{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 4 Logistic回归\n",
    "### Logistic回归的一般过程\n",
    "### （1）收集数据：采用任意方法收集数据。\n",
    "### （2）准备数据：由于需要进行距离计算，因此要求数据类型为数值型。（有限）另外，结构化数据格式则最佳。\n",
    "### （3）分析数据： 采用任意方法对数据进行分析。\n",
    "### （4）训练算法：大部分时间将用于训练，训练的目的是为了找到最佳的分类回归系数\n",
    "### （5）测试算法：一旦训练完成，分类将会很快。\n",
    "### （6）使用算法：首先，我们需要输入一些数据，并将起转化成对应的结构化数值。接着，基于训练好的回归系数既可以    对这些数值进行简单的回归计算，在这之后，我们就可以在输出的类别上做一些其他分析工作。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4.1 基于Logistic回归和Sigmoid函数分类\n",
    "### Logistic回归\n",
    "### 优点：计算代价不高，易于理解和实现。\n",
    "### 缺点：容易欠拟合，分类精度可能不高。\n",
    "### 使用数据类型：数值型和标称型数据。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 梯度上升法的伪代码\n",
    "    每个回归系数初始化为1\n",
    "    重复R次：\n",
    "        计算整个数据集的梯度\n",
    "        使用alpha×gradient更新回归系数的向量\n",
    "    返回回归系数\n",
    "### 4-1 Logistic回归梯度上升优化方法"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "from numpy import *\n",
    "def loadDataSet():   # 加载数据集\n",
    "    dataMat = []  # 创建数据列表\n",
    "    labelMat = []  # 创建标签列表\n",
    "    fr = open('testSet.txt')  # 打开测试数据文件\n",
    "    for line in fr.readlines():  # 读取文件每一行\n",
    "        lineArr = line.strip().split()  # 除去每一行的空格后拆分成列表， 例如 ['0.317029', '14.739025', '0']\n",
    "        dataMat.append([1.0, float(lineArr[0]), float(lineArr[1])])  # 增加子列表，[1.0,lineArr第一个元素（float）,lineArr第二个元素（float）]\n",
    "        labelMat.append(int(lineArr[2]))  # 添加元素，元素来源于 lineArr列表的第三个元素（transform str to int first）         \n",
    "    return dataMat, labelMat  # 返回数据列表，标签列表\n",
    "\n",
    "# 注：[1.0, float(lineArr[0]), float(lineArr[1])] 中的1.0 表示的是特征X0 的值，一般默认X0 = 1.0（or 1）\n",
    "def sigmoid(inX):   # 定义sigmoid函数， 同书P74\n",
    "    return 1.0/(1+exp(-inX))\n",
    "\n",
    "def gradAscent(dataMatIn, classLabels):  # 定义梯度上升函数\n",
    "    dataMatrix = mat(dataMatIn)   # 把数据列表转化成矩阵形式（列表和矩阵的简单差别：列表一般是一行，以逗号分隔，而矩阵是多行，没逗号）\n",
    "    labelMat = mat(classLabels).transpose()  # 把标签列表转化成矩阵形式，然后转置 （行向量 -> 列向量）\n",
    "    m,n = shape(dataMatrix)   # 取 数据矩阵的行和列   第一个是 行数m=100， 第二个是 列数n=3 \n",
    "    alpha = 0.001   # 学习率初始化 = 0.001\n",
    "    maxCycles = 500   # 最大循环次数 = 500\n",
    "    weights = ones((n,1))  # 权重初始化为 = 0.0   列向量（形式为n行1列）\n",
    "    for k in range(maxCycles):  \n",
    "        h = sigmoid(dataMatrix * weights)  # 1.0/(1+exp(-Z)    f[x,y] = Z = dataMatrix * weights\n",
    "        error = (labelMat - h)\n",
    "        weights = weights + alpha * dataMatrix.transpose() * error  # 更新权重  \n",
    "    return weights  # 返回 权重"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[ 4.12414349]\n",
      " [ 0.48007329]\n",
      " [-0.6168482 ]]\n"
     ]
    }
   ],
   "source": [
    "dataArr, labelMat = loadDataSet()\n",
    "dataMatrix = mat(dataArr)\n",
    "#print(dataMatrix)\n",
    "res = gradAscent(dataArr,labelMat)\n",
    "print(res)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4-2 画出数据集和Logistic回归最佳你和直线的函数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "def plotBestFit(weights):\n",
    "    import matplotlib.pyplot as plt\n",
    "    dataMat, labelMat = loadDataSet()  # 赋值数据列表， 标签列表（元素只有1 和 0）\n",
    "    dataArr = array(dataMat)  # array(dataMat) 不等于 mat(dataMat),但是形式上好像一样\n",
    "    n = shape(dataArr)[0]  # n=100  数组的行n=100，（数组的列m=3）\n",
    "    xcord1 = []  \n",
    "    ycord1 = []\n",
    "    xcord2 = []\n",
    "    ycord2 = []\n",
    "    for i in range(n):  # 遍历 （0->100-1）\n",
    "        if int(labelMat[i]) == 1:   # 如果第i个元素的标签值是 1\n",
    "            xcord1.append(dataArr[i,1])   # 把对应数据数组（形似矩阵）的第i行的第2个元素 增加到 xcord1\n",
    "            ycord1.append(dataArr[i,2])   # 把对应数据数组（形似矩阵）的第i行的第3个元素 增加到 ycord1\n",
    "        else:   # 如果标签列表（labelMat）中第i个元素的标签是 0\n",
    "            xcord2.append(dataArr[i,1])   # 同上 \n",
    "            ycord2.append(dataArr[i,2])   # 同上\n",
    "    fig = plt.figure()\n",
    "    ax = fig.add_subplot(111)\n",
    "    ax.scatter(xcord1, ycord1, s=30, c='red', marker='s')\n",
    "    ax.scatter(xcord2, ycord2, s=30, c='green')\n",
    "    # 绘制拟合直线\n",
    "    x = arange(-3.0,3.0,0.1)  # x取值范围 起点-3.0， 终点3.0 步长0.1 \n",
    "    y = (-weights[0]-weights[1]*x)/weights[2]  # weight[1]是1*1的矩阵\n",
    "    ax.plot(x,y)\n",
    "    plt.xlabel('X1')\n",
    "    plt.ylabel('X2')\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY4AAAEKCAYAAAAFJbKyAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3XuU3GWd5/H3t7tzI+kKxFzoJB0SIJI0co8dlEHxoEzIuOLsODOwoszoksSjLO7q4u14GR13NOroLI5CGFgZRZ3ZVZGjQUEcD+qRkAQTLrlITIBOusmFkL6QW1+++0dVd6qrq6rr111Vv0t9Xuf06a5f/br6W13dv289z/N9nsfcHRERkVLVhR2AiIjEixKHiIgEosQhIiKBKHGIiEggShwiIhKIEoeIiASixCEiIoEocYiISCBKHCIiEkhD2AFUwsyZM33hwoVhhyEiEhubN28+5O6zSjk3kYlj4cKFbNq0KewwRERiw8yeL/VcdVWJiEggShwiIhKIEoeIiARS8cRhZveY2QEzezrr2GfMbJ+Zbcl8rCzwvSvMbKeZ7TKzj1Y6VhERGV01WhzfAlbkOf5Vd78487E+904zqwf+GbgWaAFuMLOWikYqIiKjqnjicPdHgcNj+NZWYJe773b3k8D3gevKGpyIiAQW5hjHB8zsyUxX1hl57p8HtGXd3ps5JiIiIQorcXwTOAe4GOgAvpLnHMtzrOA+t2a2ysw2mdmmgwcPlidKkRhr62zjlvW30HpXK7esv4W2zrbRv0mkBKFMAHT3/YNfm9ldwE/ynLYXaM66PR9oL/KY64B1AMuWLdNG6lLT2jrbuOiOi+g52UPvQC9bXtzCfU/dx9Y1W2me3jz6A4gUEUqLw8yasm7+OfB0ntM2AovNbJGZTQSuBx6oRnwicbf2t2uHkgZA70AvPSd7WPvbtSFHJklQ8RaHmX0PuAqYaWZ7gU8DV5nZxaS7np4DVmfOnQv8i7uvdPc+M/sA8HOgHrjH3Z+pdLwiSbBh34ahpDGod6CXx/c9HlJEkiQVTxzufkOew3cXOLcdWJl1ez0wolRXRIpbPm85W17cMix5TKibQOu81hCjkqTQzHGRBLrtituYNnEaE+omAOmkMW3iNG674raQI5MkUOIQSaDm6c1sXbOV1ZetpnVuK6svW62BcSmbRC6rLpIUbZ1trP3tWjbs28Dyecu57YrbSr74N09v5vaVt1c4QqlFShwiEaWSWokqdVWJRJRKaiWqlDhEIkoltRJVShwiEZK9TMjxvuM02PDe5KiU1Go5k9qmMQ6RiMgd02ioa6Df+2mwBvq8LzIltRp7EbU4RCIid0yjb6CPhroGls5aGqmSWo29iFocIhFRaExjSsMUNty8IaSoRtLYi6jFIRIRy+ctH5rpPSgqYxrZ4hKnVI4Sh0hExGWZkLjEKZWjxCESEXFZJqQacapqK9rMPXl7Hi1btsw3bdoUdhgiMga5VVuDLZooJtEkMbPN7r6slHPV4hCRSFHVVvQpcYhIpKhqK/qUOEQkUlS1FX1KHCKSV1gD1JWs2tKge3locFxERgh7gHpwH5Jfv/BrBnyAOqvjygVXBtqPJN9jatC9sEgNjpvZPWZ2wMyezjr2JTPbYWZPmtmPzOz0At/7nJk9ZWZbzEyZQKRKwh6gbp7ezG1X3MYLnS+w49AOtu7fyp2b7+SiOy4acysh7OeUJNXoqvoWsCLn2MPAa9z9QuAPwMeKfP+b3P3iUjOhiAw3lu6ZKAxQl/tCH4XnlBQVTxzu/ihwOOfYQ+7el7n5GDC/0nGI1KLB7pk7N9/JxvaNJb9rj8IAdbkv9FF4TkkRhcHx9wAPFrjPgYfMbLOZrSr2IGa2ysw2mdmmgwcPlj1IkTj65C8/yZHjRwK/a4/CsiLlvtBH4TklRaiJw8w+AfQB9xU45Qp3vxS4Fni/mb2h0GO5+zp3X+buy2bNmlWBaKXSVPFSXm2dbXznqe/gDC+AKeVdexSWPyn3hT4KzykpqlJVZWYLgZ+4+2uyjt0ErAGudvejJTzGZ4Aed//yaOeqqip+VPFSfresv4VvbPwGAwwMO24Y777w3TROamTDvg0sn7d8XNVKlTRYXfX4vsdpndca2TiTIEhVVSj7cZjZCuAjwBsLJQ0zmwrUuXt35utrgM9WMUypomIDobevvD3k6OJpw74NI5IGpBPH/Tvv52jv0cjv4Nc8vVmvfwRVoxz3e8DvgPPMbK+ZvRf4OtAIPJwptb0jc+5cM1uf+dY5wG/MbCvwOPBTd/9ZpeOVcKjipfzyjRHUUcfiVy0eShqgslQJruItDne/Ic/huwuc2w6szHy9G7iogqFJhCyft5wtL24ZljyiXPEy2IUS5a6e2664jfueum9E99/khslK0jIuUaiqEolVxctYS1yrrdBg8JULrlRZqoyLlhyRyIjLQOgt62/hzs13jmgdrb5sdSz646NUiJDbcrvxwhv5zpPfiXRLLqmCDI4rcUjNaets45O//CQP7noQDK4991o+96bPlXyBar2rlY3tG0cen9vKhps3lDvciohCks5NYA3WQL/3U19XT99AnyrrqizyVVUiYWnrbOOCb15A54nOoWP3br2X+3fcz1Pve6qkC1TcxmPyiUK1Um4lXV9mMYm+gfRnVdZFl8Y4JBFKnTy49rdr6TrRNeJ494nukquK4jQeE2X5KulyadA+mtTikNjL7fIoNi9hw74NI2ZSAwwwUPIFanDQOeyunrjL13LLFbeWXK1Qi0NiL8gqqsvnLcewEcfrqAt0gRrs6tlw8wZuX3m7ksYY5LbcGqwBw2ioS7+fVUsuupQ4JPaCTB687YrbSE1KjTjeOKmxJi5QUVoPLLdceM2yNfzuvb9jzWVrtJZUxKmrSmIvyGB18/RmnnrfU0WrqqI8uW88seV26f2+4/fc9cRdLJm5ZNy7641VvkH65fOXVzUGCU7luBJ75ZyXEKU5DuWOLd/8k0FRep4SjkhtHStSaeVcLjvK24uON7ZiVUxRep4SfeqqkkQo17yESiy2WK6ur/HGNloV02iPFeUuPKkuJQ6RLOWe3BekVLjSseUuepir2GOV83lI/KmrSmKt3FVC5Z7cV86ur/HGlt2ld9Gci5hUP6nk0tcod+FJ9anFIbFViXfB5Z7cN57upXxdQ+ONLbtLL8h6VdovRbIpcUhsVWrXwGLjJUH7+VtmtbC5ffOwnfhK6V4qlhTLtW5TkHGhJKzPJeWjriqJrWq/Cw66D0dbZxv377h/xPatp004bdTupah1DSVlfa4oTYCMMyUOia18W6NW8l1w0Iv52t+u5Wjv0WHHDOPt57191O6lqHUNlbPkOSxx2YArDqqSOMzsHjM7YGZPZx2bYWYPm9mzmc9nFPjemzLnPGtmN1UjXomHar8LDnoxz3e+42w/tH3Un1XtpFiKuK/PFbVWXJxVq8XxLWBFzrGPAo+4+2LgkcztYcxsBvBpYDnQCny6UIKR2lPtd8FBL+bjufgnpWsoSqLWiouzqiQOd38UOJxz+Drg3szX9wJvz/Otfwo87O6H3f1l4GFGJiCpYdV8Fxz0Yj6ei3+1kmIt9flHsRUXV1Vbq8rMFgI/cffXZG4fcffTs+5/2d3PyPmeDwOT3f3vM7c/CRxz9y8X+1laq0oqJeiWq1HYorWQKK/LVQm19nyDStLWsSM3TiDPLjyAma0CVgEsWLCgkjFJDQu6tEk1tmgd61IglSpnjiptwFU+YSaO/WbW5O4dZtYEHMhzzl7gqqzb84Ff5Xswd18HrIN0i6O8oYpE03gmQUahz7+UpFfONbKisNd6EoSZOB4AbgK+kPn84zzn/Bz4X1kD4tcAH6tOeBKEFsALR6FWw59998+Y3DC56GtRaNHDY33HaOtsq/jrV0rS0xpZ0VStctzvAb8DzjOzvWb2XtIJ4y1m9izwlsxtzGyZmf0LgLsfBj4HbMx8fDZzTCJE9fHhKdRqeOrAU6O+FoOD94PrVQ3afnB7VV6/UspjVUIbTdWqqrrB3ZvcfYK7z3f3u939JXe/2t0XZz4fzpy7yd3/a9b33uPu52Y+/k814pVg9M8dnnyVQtkKvRaDLcTm6c2kJg7fSrfP+6ry+pXSVfboC4+G3p0mI0V9cFxiIAp95bVqtKXSYeRrkdv9U8r3VMJo61+1dbax89DOEd/XYA0qoQ2ZlhyRcVN9fHhy53tcMPsCGmz4+8Hc1yK3hZhPNV6/0ea5rP3tWvq9f8T31dfVayJkyJQ4ZNwKXQBuvPDGWE8ui8LkuFJiyJ4E+dP/8lMaJzUWnXRYbAvZQt9TCaNNctywbwN9A30jvm/JzCUaGA9Z1SYAVpMmAFZf7kS3Gy+8kWvvuza2k62iMFlsrDGMNunwlvW3cOfmO0d0ES2ZuYQpDVMiM7+hUJyrL1utktoKCDIBUIlDKiLu//RRiL9SMUQhKZYiLnEmRZJmjktMxX3APArxVyqGuMygjkuc1XTsZD8793ezrb2LpU2NXLIgnDVflTikIuK+Y1wU4i81hrFMvozLDOq4xFkJB7qPs70jnSS2dXSxrb2TPYdeYSDTSbT6jWeHljjUVSUVEfduhijEX0oMUYhTxqd/wNlzqIdn2rvSiaKji23tXRzqOTF0zrzTp7C0KUXL3BQtTSnOn5ti/hlTMMu3nN/YaIxDiSMSorwybCmiEP9YB7rjMpZUa3pO9LGjo4vtHZlWREc3O1/s4nhvenvhCfXG4tmNQwliaVP68/TTCk/yLBclDiUOqRGtd7WysX3jyONzW9lw84YQIhIAd6ej83g6QWS6mrZ3dPHcS6e2Ej79tAlDyeH8uenP58yaxsSGgLMkUino7h55vLERurpKfhgNjovUiLKMxZTpwlOrevsH2HWgZ1iS2NbRxZGjp16Tha86jaVNKf7zpfOHkkTT9Mnl6WrK99oVO14GShwiMZa75MiYJu+FcOGJq86jvUOJYTBR7DrQw8n+dFfTpIY6ljSluPY1Zw61JpY0pZg2KVmX2mQ9G5Eao5LVynB39r58jGfahyeJfUeODZ0zc9pEWuZO58pXz6QlMxaxaOZUGuqTvyCHxjhEKiBW+5MU6y5J4PUh1/Hefp7d38O2js6h8tftHV10n0gvd1JnsGjmVM6fO32osmlpUyOzGyeHHHlGmV4/jXGIhEibD0XXSz0nMiWvnUPjEX88+Ar9mckRUyfWs6QpxdsvmTdU2fTqOY1MmVgfcuTRosQhUma1tpd3FPUPOM+/9MrQnIjB8tf9XafmRjRNn0xLU4prWs4cShILZpxGXV355kZURWNj4eKGClHiECmzKCxXEkgIF55yOnqyjx0vdg8re93R0c2x3vSS7A11xrmzp3HFuafGIpY2pThj6sSQIy+TECrflDhEyiwKy5UEEpOSW3fnQPeJYSWv29u72PPSK0Nd+Y2TG2hpSnF9a/NQglh8/iImdb488gFVbjxmoSUOMzsP+LesQ2cDn3L3r2WdcxXwY2BP5tAP3f2zVQtSZAzKUiKbAOMpEOjtH2D3wVeGDVhv6+ji8Csnh85pnjGFlqYUb7t47lCSyLsMR76kASo3HodIVFWZWT2wD1ju7s9nHb8K+LC7vzXI46mqSsIWheVKwhRkDa2u473s6OhmW3tnpqupm537uznZl54bMbGhjlfPmTbUzdQydzpLmhpJTS5xGY4arxorVRyrqq4G/pidNETirJZXdYUCBQInevjMI/+bvz7vQ6fmRnR00Xb41NyIGVMnsrSpkb95/UKWNjXS0jSds2dNZUINzI2Ik6gkjuuB7xW473VmthVoJ936eKZ6YYnIWDy2dxP0NTN1YBETB85mop/NhIFFPPL4NB55fDNmsOhVU7lw/ulc/9oFmZZEitmNk8q64qtURuhdVWY2kXRSON/d9+fclwIG3L3HzFYC/+Tuiws8zipgFcCCBQsue/55NV5EquHlV05mrfaaLn/dub8T93QrYYDj9Npz9NU/z+UL5/N319zMkjMbOW1ild63qquqJLFaHdfMrgPe7+7XlHDuc8Aydz9U7DyNcSRXrGZkJ8zAgPPC4aMj5kZ0dB4fOmdOalJ6kHoGrHvy0/QM7OCYv8CE+vrw9gnRIo4lidsYxw0U6KYyszOB/e7uZtYK1AEvVTM4iQ7NyB6HgBfP47397Hzx1KZC2zq62NHRxSsn03Mj6uuMs2dOZfmiGZklONIfM6dNGnqM1Veti0aBgJJD2YWaOMzsNOAtwOqsY2sA3P0O4B3A+8ysDzgGXO9hN5EkNJqRPQ5FVsDN3qJ0sBWx+2DP0Bal0yY1sLSpkXdcNj8zw3o6iy84m8lHDo98vKxEVOsFAkkWauJw96PAq3KO3ZH19deBr1c7rsRIWBM9djOyK2Ecr2m/1bFnxlyemX0222cvYtvss9k2+2wOff6RoXMGtyhdeUHTUPnr/DOmjFyGI1/SAM2NqBFR6KqSSknYPguxm5FdCSW+pj0n+tj5YtbGQu/6CjtnncXxCekVXSf097L40AtctWcTSz9261CSqMYWpRJ/ShwSG5qRPZIDHY0z0y2IR54tvEXpyeO8c8vPWHpgNy3793DuS21MHEgvG85Pv5b/wUUKCL2qqhJUVZWRwDLEWp6R3ds/wK6mc9iW3dU0ZxFHpqSGzhncorRlaN+IzBaldUUm0I3lbyGBf1u1LlbluJWgxJGhf+4xC7vsd3CL0sHB6u0dXTy7P2uL0t4TnHfoeVoO7KFl/25aDuzmvF1baSy0DEe5x7v0txVcxMcclTiUONL0zz0mQdZZGq/cLUoHK5uGb1E6Kb38xtwULR9aQ8uBPSw6vI8GH8h9sLLGVlTEL4KRFPH/x7jN45BKifk+C2GpVNnv4Bal2zuGJ4ncLUovWXA6N15+Vv4tSv/699F4TZUcapoSR5Lpn3tMylH2+1LPiVNdTXm2KD1tYj1Lm1Jcd8lcljalOH/udM4rZYtSvaYSAUocIjmClP0G3aI0nSRK2KI0CV1BSXgOkpfGOCQywh6Qzo4j3xjHY+/5PT3HUsMSxM4Xuzl6cvgWpdkVTUubUswYyxalEe8PL0kSnkM5Rfz3UbbB8czqtLPc/Y85xy909yfHF2blKHHETzUHpEfj7mzZt4d/+NW32dbeSaq+hfr+s9j78slhW5QOth4GE8W5s6cxqWGUrqZSBb3IRPHdfcQvlFUXxdcoS1kGx83sr4CvAQfMbALwN+6+MXP3t4BLxxuoyKCw1qHq6x9g96FXhrUitrV38dIrJ4H0/9AZM6aw9MwUf3HpqSQx7/Q8W5SGKWGrBCRSBJJDuRQb4/g4cJm7d2RWpv22mX3c3X8IROg/RpKgGutQZW9Rur0jvfJr7hal581p5M1L52TKX4tsURrxd4+hS6VGP0diq1jiaHD3DgB3f9zM3gT8xMzmk17pQKRsyrkOlbuz78ixoRVft3V05t2itKUpNfYtSvUOvzj9HhKtWOLoMrNzBsc3Mi2Pq4D7gfOrEZzUjrGuQ3Wyb4BnDwwuCd6dThLtXXQdT8+NyN2idGlTI+fPnR79LUqTPAcnCc+hxhVLHB8hp0vK3bvNbAXwsYpGJTWneXozW9dsLboOVb4tSncd6KEvMzdiyoR6ljQ18taL5g6NRVR1i9JySnJ3V7Wem7oTK6ZgVZWZ7QbuBL7i7n2ZY3OArwDnuftrqxZlQKqqireBAaft5aOnlgTPDFy3Z21ROrtx0lBV09JMklj4qqnUF5sbUU5RqxiK2kUyCr+fKMQQI+VacuQy4B+A35vZrcAFwP8A1gLvHneUIsCxk/3s3N89NMN6e2a2dfYWpefMmsprF80YNj8ie4tSQe+gpaoKJg53fxlYk0kavwDagcvdfW+1gpNkOdh9YsQM60JblA4uw7F4zjQmTyjT3IhySvIYRDno95NoxeZxnA58EVgOrABWAg+a2a3u/ssqxScx1D/g7DnUw7acfawPdp9ahmP4FqXpqqa8W5RGld7hF6ffT6IV66p6AvgG8P7MGMdDZnYx8A0ze97dbyhHAGb2HNAN9AN9uX1sli59+SfSieso6YmIT5TjZ8v4jdiitKObnS92cbw3PTdiQr2xeHYjb3z1LFoyS3Boi1IZs2qM5URtvCiCiiWON+R2S7n7FuD1ZnZzmeN4k7sfKnDftcDizMdy4JuZz1JF7k5H5/FTYxGZZJG9Ren0KRNoaUrxzuVnDY1HnDNrGhMbSpwbIbVjrBfnIPNnxtpdpjk6oyo2xlFwLMPd76pMOHldB/yrp8u/HjOz082saXByopRfb/8Auw70DF+Go6OLI0dPTc4b3KL0Ly6dP1TV1DR9crTnRkh0VOPirNZBxUShwN1Jd4M5cKe7r8u5fx7QlnV7b+bYsMRhZquAVQALFiyoXLQJ03msd9ieEYNzI4a2KG2oY8mZjaw4/8z0DnRNKZY0pZg2KQp/OhGnLg9JqCj891/h7u1mNht42Mx2uPujWffnews7ogg7k3DWQXoeR2VCjS93p+3wsaHWw2CyGL5F6USWNqW48tUL011NTSkWzZxKQ6nLcFRCnC++6vKQhAo9cbh7e+bzATP7EdAKZCeOvUD2utrzSZcGSwGDW5Ru6+gcWq8p3xall551BjdeftbQftbDtiiNCl18RSIn1MRhZlOBusxSJlOBa4DP5pz2APABM/s+6UHxTo1vnPJSz4mhNZoGk8Sugz15tyhtaZpOy9xUaVuUikRRNeaHaA7KqMJuccwBfpQZUG0AvuvuPzOzNQDufgewnnQp7i7S5bh/G1KsocrdonSwuyl7i9IzU5M5f26Kt7TMGZphfdZoW5RKvLvD4mqsF+dqvB56zUcVauJw993ARXmO35H1tQPvr2ZcYTt6so8dL3YPG7TOt0Xp68+ZObQD3dKmFGeMZYtSUXdYGHRxjrWwWxw1r/NYL0+88PKw0tc9h14ZtkVpS1OKv1rWPFTVtHhOGbcolcpRl4cklBJHyDY9d5j33pteybd5xhRamlK87aK5Qyu/Rm6L0mqL88VX76oloZQ4QvbaRTP499WvK7xFaa2L68VX4yaSYEocIUtNnkDrohlhhyHlpnETSTAtIiQiIoEocYiISCBKHCIiEogSh4iIBKLEIVIJhcqFq1VGnEqB2ciPVKo6P18STVVVIpUQdsmtqrqkgtTiEClX60Dv8qVGKHGIdHWB+8iPoK0GvcsvTRQTbBRjijAlDhGprigm2CjGFGFKHCIiEogSh0gShV3VJYmmxCEStkr0r5dr3EYkDyUOkXIZ67t89a9LzChxSH6qMgmuEu/yo/j7H+/fRhS70aIYU4SFljjMrNnM/sPMtpvZM2Z2a55zrjKzTjPbkvn4VBix1iS9C46mav3+iyWH8f5tRLEbLYoxRViYM8f7gA+5+xNm1ghsNrOH3X1bznm/dve3hhCfSO3SGwcpIrQWh7t3uPsTma+7ge3AvLDiERGR0kRijMPMFgKXABvy3P06M9tqZg+a2flVDUykGtSPLjETeuIws2nAD4APuntuh+ITwFnufhFwO3B/kcdZZWabzGzTwYMHKxewSLnl61+vVSrKiIVQE4eZTSCdNO5z9x/m3u/uXe7ek/l6PTDBzGbmeyx3X+fuy9x92axZsyoad01QlUm4ovz7r2RsGluJhdAGx83MgLuB7e7+jwXOORPY7+5uZq2kE91LVQyzdqmaJFxh//4bG/NfrBsbw49NQhdmVdUVwLuAp8xsS+bYx4EFAO5+B/AO4H1m1gccA653r+V2vEiZFCqrHUwMSg5SRGiJw91/A9go53wd+Hp1IhKpIeoSknEIfXBcRETiRYlDkkeVOfEV5aIAGaLEIcmjbpjqK1ey1tIfsaDEISLjp2RdU5Q4RGqRuoRkHMIsxxWRsKjrR8ZBLQ6RYjTQLjKCEockTzm7YdR3LzKCuqokedQNU33FliiRxFGLQyRuoth9pjLamqLEIRI36j6TkClxiIhIIEocIsVovoPICEocIsWo7768ojg+I4EpcUg8JO2Ck7TnUyqNzySCEofEQ9IuOON5Puo+k5BpHodI3KibTEKmFocMV6tdKCJSslATh5mtMLOdZrbLzD6a5/5JZvZvmfs3mNnC6kdZY5LWJZQkSuoSEaElDjOrB/4ZuBZoAW4ws5ac094LvOzu5wJfBb5Y3Sil7HTxG7skJHWNzyRCmC2OVmCXu+9295PA94Hrcs65Drg38/X/A642M6tijFJuY734Je2Ck7TnUyqVNydCmIPj84C2rNt7geWFznH3PjPrBF4FHKpKhBIdSbuwJO35SE0Js8WRr+XgYzgnfaLZKjPbZGabDh48OO7gREQkvzATx16gOev2fKC90Dlm1gBMBw7nezB3X+fuy9x92axZsyoQbo2o1S4UESlZmIljI7DYzBaZ2UTgeuCBnHMeAG7KfP0O4JfunrfFIWWiPujoUlKXiAhtjCMzZvEB4OdAPXCPuz9jZp8FNrn7A8DdwLfNbBfplsb1YcUr45RKFR8A18VvdEreEhGhzhx39/XA+pxjn8r6+jjwl9WOSyqgWNJQI1IkVjRzXCQuNAdGIkKJQyQukjABUBJBiUNERAJR4hARkUCUOKQ6VEoqkhjaj0OqQ6WkIomhFodIXKjVJhGhFodIXKjVJhGhFocIaI6ESABKHCKgORIiAShxgN5tSuXob0sSSIkD9G6z1lXyIq6/LUkgJQ4RXcRFAlHiEBGRQJQ4RIrRHAmREZQ4RIrR3AmREZQ4QDNypbDxVkXpb0sSSDPHQe8qa11jY/4B8kLHofQBdf1tSQKpxSHxUMn5EF1d6e1rcz900RfJK5QWh5l9CfhPwEngj8DfuvuRPOc9B3QD/UCfuy+rZpwSIZoPIRIZYbU4HgZe4+4XAn8APlbk3De5+8VKGiIi0RBK4nD3h9y9L3PzMWB+GHGIiEhwURjjeA/wYIH7HHjIzDab2apiD2Jmq8xsk5ltOnjwYNmDlBqlqiiRESo2xmFmvwDOzHPXJ9z9x5lzPgH0AfcVeJgr3L3dzGYDD5vZDnd/NN+J7r4OWAewbNkyH/cTEAENkIvkUbHE4e5vLna/md0EvBW42t3zXujdvT3z+YCZ/QhoBfImDkm4YiWzIlJVoXRVmdkK4CMHNWcPAAAFEUlEQVTA29z9aIFzpppZ4+DXwDXA09WLUiJFJbMikRHWGMfXgUbS3U9bzOwOADOba2brM+fMAX5jZluBx4GfuvvPwglXREQGhTKPw93PLXC8HViZ+Xo3cFE145IESaUKd22plSIyLlGoqhIpP00YFKkYJQ4REQlEiUNERAJR4hARkUCUOEREJBAlDkkmLRUiUjHayEmSSSW3IhWjFoeIiASixCEiIoEocYiISCBKHCIiEogSh4iIBGIFtsKINTM7CDwfdhwBzAQOhR3EGCju6oljzKC4q2m8MZ/l7rNKOTGRiSNuzGyTuy8LO46gFHf1xDFmUNzVVM2Y1VUlIiKBKHGIiEggShzRsC7sAMZIcVdPHGMGxV1NVYtZYxwiIhKIWhwiIhKIEkdEmNnnzOxJM9tiZg+Z2dywYyqFmX3JzHZkYv+RmZ0edkyjMbO/NLNnzGzAzCJfOWNmK8xsp5ntMrOPhh1PKczsHjM7YGZPhx1Lqcys2cz+w8y2Z/4+bg07plKY2WQze9zMtmbi/ruK/0x1VUWDmaXcvSvz9X8DWtx9TchhjcrMrgF+6e59ZvZFAHf/SMhhFWVmS4EB4E7gw+6+KeSQCjKzeuAPwFuAvcBG4AZ33xZqYKMwszcAPcC/uvtrwo6nFGbWBDS5+xNm1ghsBt4eg9+1AVPdvcfMJgC/AW5198cq9TPV4oiIwaSRMRWIRUZ394fcvS9z8zFgfpjxlMLdt7v7zrDjKFErsMvdd7v7SeD7wHUhxzQqd38UOBx2HEG4e4e7P5H5uhvYDswLN6rReVpP5uaEzEdFrx9KHBFiZp83szbgncCnwo5nDN4DPBh2EAkzD2jLur2XGFzM4s7MFgKXABvCjaQ0ZlZvZluAA8DD7l7RuJU4qsjMfmFmT+f5uA7A3T/h7s3AfcAHwo32lNHizpzzCaCPdOyhKyXmmLA8x2LRGo0rM5sG/AD4YE5PQGS5e7+7X0y6xd9qZhXtHtQOgFXk7m8u8dTvAj8FPl3BcEo2WtxmdhPwVuBqj8igWYDfddTtBZqzbs8H2kOKJfEyYwQ/AO5z9x+GHU9Q7n7EzH4FrAAqVpigFkdEmNnirJtvA3aEFUsQZrYC+AjwNnc/GnY8CbQRWGxmi8xsInA98EDIMSVSZpD5bmC7u/9j2PGUysxmDVYzmtkU4M1U+PqhqqqIMLMfAOeRrvZ5Hljj7vvCjWp0ZrYLmAS8lDn0WNSrwczsz4HbgVnAEWCLu/9puFEVZmYrga8B9cA97v75kEMalZl9D7iK9Iqt+4FPu/vdoQY1CjP7E+DXwFOk/w8BPu7u68OLanRmdiFwL+m/jzrg3939sxX9mUocIiIShLqqREQkECUOEREJRIlDREQCUeIQEZFAlDhERCQQJQ6RCsistLrHzGZkbp+RuX2Wmf3MzI6Y2U/CjlNkLJQ4RCrA3duAbwJfyBz6ArDO3Z8HvgS8K6zYRMZLiUOkcr4KXG5mHwT+BPgKgLs/AnSHGZjIeGitKpEKcfdeM/ufwM+AazLLoovEnlocIpV1LdABxGIzI5FSKHGIVIiZXUx6577Lgf+e2WFOJPaUOEQqILPS6jdJ7+nwAukB8S+HG5VIeShxiFTGzcAL7v5w5vY3gCVm9kYz+zXwf4GrzWyvmUV2ZV6RfLQ6roiIBKIWh4iIBKLEISIigShxiIhIIEocIiISiBKHiIgEosQhIiKBKHGIiEggShwiIhLI/wc2b34RsambOgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from numpy import *  ### why do in this way ???\n",
    "# import numpy\n",
    "dataArr, LabelMat = loadDataSet()\n",
    "weights = gradAscent(dataArr,LabelMat)\n",
    "plotBestFit(weights.getA())  # #将numpy矩阵转换为数组\n",
    "# what is getA() ???  结果如下\n",
    "# print(type(weights))  <class 'numpy.matrixlib.defmatrix.matrix'>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4-3 随机梯度上升算法\n",
    "#### #注：和梯度上升法的区别：第一，后者的变量h和error都是向量，而前者则全是数值；第二，前者没有矩阵的转化过程，所有变量的数据类型都是Numpy数组"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [],
   "source": [
    "def stocGradAscent0(dataMatrix, classLabels):\n",
    "    m,n = shape(dataMatrix)  # 取数组的行m， 列n\n",
    "    alpha = 0.01  # 学习率初始化\n",
    "    weights = ones(n)   # 创建 含有 n 个元素的 数组\n",
    "    for i in range(m): # 循环每一行的元素\n",
    "        h = sigmoid(sum(dataMatrix[i]*weights))   # 1.0/(1+exp(-Z)   # h 是向量\n",
    "        error = classLabels[i] - h \n",
    "        weights = weights + alpha * error * dataMatrix[i]    # 更新权重\n",
    "    return weights   # 返回权重"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY4AAAEKCAYAAAAFJbKyAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xl8VPXV+PHPCQlhC2FJ2EJiEEIQERBiUCkoLoDUve5rtRWx7lXp9uvTPu3T11ODuKIC7lbcumh9rLKpCCiyKYgKgbAGwhaQEAhkPb8/MmAIWWaSmbn3zpz365UXmTt3Zs5kLvfM/Z7vIqqKMcYY468YpwMwxhjjLZY4jDHGBMQShzHGmIBY4jDGGBMQSxzGGGMCYonDGGNMQCxxGGOMCYglDmOMMQGxxGGMMSYgsU4HEApJSUmanp7udBjGGOMZy5cvL1TVZH/2jcjEkZ6ezrJly5wOwxhjPENENvu7rzVVGWOMCYglDmOMMQGxxGGMMSYgljiMMcYEJOSJQ0ReFJFdIvJNjW1/FJFtIrLC9zOunseOFZFcEckTkV+HOlZjjDGNC8cVx8vA2Dq2P6aqg30/H9S+U0RaAE8DFwD9gWtFpH9IIzXGGNOokCcOVZ0P7G3CQ7OBPFXdoKplwJvAJUENzhhjTMCcrHHcJSJf+5qyOtZxfwqQX+P2Vt+2OonIeBFZJiLLdu/eHexYjfGc/KJ87v7gbrKfy+buD+4mvyi/8QcZ4wenEsezQG9gMLAdmFzHPlLHtnoXSFfV6aqapapZycl+DX40JmLlF+UzaOogpi2fxtKCpUxbPo1BUwdZ8jBB4UjiUNWdqlqpqlXAc1Q3S9W2FUitcbsnUBCO+IzxupzPcjhQdoDyqnIAyqvKOVB2gJzPchyOzEQCRxKHiHSvcfMy4Js6dlsKZIhILxFpCVwDvBeO+IzxusXbFh9NGkeUV5WzZNsShyIykSQc3XHfABYBmSKyVUR+BuSIyCoR+RoYBdzv27eHiHwAoKoVwF3ALGA18LaqfhvqeI2JBMNShhEXE3fMtriYOLJT6rq4NyYwolpv2cCzsrKy1CY5NNHsSI3jSHNVXEwc7Vq2Y+WElaQmpjb+BCbqiMhyVc3yZ18bOW5MBEpNTGXlhJXcPvR2sntkc/vQ2y1pmKCJyGnVjYkU+UX55HyWw+JtixmWMoyJwyf6ffJPTUzlqXFPhThCE40scRjjUrWbm1bsWMGMVTPsysE4zpqqjHEp61Jr3MquOIxxkZpNU5v3bXZtl9rmNKEZ77PEYYxL1G6aiqmjQcANXWqtCc1YU5UxLlG7aaqKKgDEN/vOkS61E4dPdCxGsCY0Y1ccxrhGXaO9AZLbJpOemE52SrYrmoRsVLqxxGGMSwxLGcaKHSuOOSnHxcRxVf+rXNWttr44nW5CM+FjTVXGuMTE4RNp17Ld0alC3NI0VZtX4jShY4nDGJfwymjvcMRpa4m4m81VZYxxFZtnyxk2V5UxxrOs15b7WeIwxriK9dpyP0scxhhXsbVE3M8ShzGmTk4VqEPZa8uK7sER8uK4iLwIXAjsUtUBvm2TgIuAMmA9cIuq7qvjsZuAYqASqPC3cGPFcWOax+kC9ZG5sBZsWUCVVhEjMYxIG9GsAZBOvye3c1tx/GVgbK1tc4ABqjoQWAv8poHHj1LVwf6+IWNM8zldoE5NTGXi8IlsKdrCmsI1rNy5kmnLpzFo6qAmXyU4/Z4iScgTh6rOB/bW2jbbt6Y4wBdAz1DHYUy0akrzjBsK1ME+0bvhPUUKN9Q4bgU+rOc+BWaLyHIRGR/GmIyJCIu3LibjqQymLJ3C0oKlTF021a9v7W4oUAf7RO+G9xQpHE0cIvI7oAKYUc8uw1V1CHABcKeIjGzgucaLyDIRWbZ79+4QRGuMt+QX5XPWy2dRWll6dFuFVlBcWtzot3Y3TCsS7BO9G95TpHAscYjIzVQXza/Xeir0qlrg+3cX8A5Q7xGjqtNVNUtVs5KTk0MRsgkx6/ESXDmf5RyTNI6o0IpGv7W7YfqTYJ/o3fCeIkVYphwRkXTg/Rq9qsYCjwJnqWqdlwci0haIUdVi3+9zgD+p6szGXs96VXmP9XgJvuznsllasLTO+24eeDMJ8QmuX8HvSO+qJduWuGZa+UgVSK+qkE+rLiJvAGcDSSKyFfgD1b2o4oE5IgLwhapOEJEewPOqOg7oCrzjuz8WeN2fpGG8qaFCqJumFPeSYSnD+GrHV1RUVRyzvWVMS97NfZeS8hLXr+CXmphqn78LhaNX1bWq2l1V41S1p6q+oKp9VDXV1812sKpO8O1b4EsaqOoGVR3k+zlZVf8S6liNc7zW48ULzWoTh08koWUCsTE/fD+MbxHPxZkXH00aYN1STeDc0KvKGE/1eDnSrDZt+TSWFixt9viCUDnSpj9h6ASye2Rz12l3se7udWwu2uypJG3cx1YANK4wcfhEZqyacVyNw409XrzUrFZXU4+bVvA7UsM4Umu5YeANvPb1a66vvUQ7W4/DuEa4CqH5Rfn8/uPf82HehyBwQZ8L+POoP/v9WvUVnbN7ZLP4tsXBDjfo3NIRoXYcsRJLpVbSIqYFFVUV1kEizFxVHDfGX+EohOYX5XPKs6dQVFp0dNsrK1/h3TXvsuqOVX6doNz0jb0pjjRhOd1bqfaVW4VvMokjxXw3X8lFO6txmIjgb7E657Mc9pfuP267P4PijoiEgWRHkvTi2xbz1LinHPlGX1eHiNqs9uJOdsVhPK92k0dD3UsXb1uMcnzzbBVVfp+g3PKN3evqunKrzUtXctHErjiM5wUyGd6wlGEIctz2GGICOkG54Ru719W+couVWAQ52n3Yi1dy0cISh/G8QMaATBw+kfbx7Y/bnhCfcPQE5eYxGs2NzU3vrfYUIBOyJrDoZ4uOdh+2KUHcy3pVGc+7+4O7mbZ82nHF6tuH3l5nUbWhXlVu6XFUl+bGVlcvphYxLeiX1K/ZiyQZ73PbQk7GhFSgxerUxFRevuxldj60k50P7uTlS18+esJ082I/zY2trl5MpZWlQVkkyUQXSxzG84I562kopj4JVvNQc2NrqBeTmxKkcT/rVWUiQrDGgAR7jEYgPb5CHVtjvZgaS0K1R3lb01b0sisO42nBLvYGe4xGMJu+mhtb7cfX1lAS8sr8XCY8LHEYzwrFySzYi/00p3mpdlIEmhVbzfc2qOsg4lvE+9311c21HxN+1lRlPCtUkw021OwVaHNN/+T+LC9YThVVR7f507zUUBNXsN5bIHODeW3aexNaljiMZ4X7ZBZovSK/KJ9317x7TNIAaBPXptHmpXDMwBtIXcjr83MdYXWa4LCmKuNZ4V7DI9DmmpzPcigpLzlmmyBcmnlpoycrt33Dj4T5uaxOEzxhSRwi8qKI7BKRb2ps6yQic0Rkne/fjvU89mbfPutE5OZwxGu8Idwns0BP5nXtryirC1c3+lpuW9gq2LUfJ1idJnjCdcXxMjC21rZfAx+pagbwke/2MUSkE9VrlA8DsoE/1JdgTPQJ98ks0JN5c07+bvyG7/X5udx2FedlYUkcqjof2Ftr8yXAK77fXwEureOhY4A5qrpXVb8H5nB8AjJRLJwns0BP5s05+YcrKbpp7qpQc9tVnJeFba4qEUkH3lfVAb7b+1S1Q437v1fVjrUe8yDQSlX/x3f798AhVX2kodeyuapMqAS6SmE4VjVsasHXzfNyhUK0vd9ABTJXldsTx0NAfK3EUaKqk+t4/vHAeIC0tLShmzdvDtVbMcY1mnMyDHRyyEgQruWJvcgrkxzuFJHuAL5/d9Wxz1ag5qfaEyio68lUdbqqZqlqVnJyctCDNQ2LpiYPN6mv4Pvj13/c6GdRX5v/29+9HbbPz5/jJpjHltfrNG7h5BXHJGCPqv5VRH4NdFLVibUe0wlYDgzxbfoSGKqqteslx7CmqvCyJgDnZD+XzdKCpfXe39BnUdcVB1R3Ge7QqkPIPz9/jhs7tuqnqogcvyhZU7nuikNE3gAWAZkislVEfgb8FThfRNYB5/tuIyJZIvI8gC9B/BlY6vv5U2NJw4SfdXN0Tl0F35oa+iyOFO9jap0GFA3L5+fPcWPH1vFWbS3ixhcW8/7X2x2LISwjx1X12nruOreOfZcBP69x+0XgxRCFZoLAujk6Z+LwicxYNeOYk2ttdX0WR9r6UxNTOVRxiMMVhxt9TLD5c9zM3zLfji2fDbsPMHnOWv7z9XY6tonj0sEpjsViU46YZouU6Si86Ei33SMF30MVh1i9ezUVWnF0n9qfRe3mn9pXHHU9JhQaO27yi/LJLcw97nGxEhtVx9aOosM88dE63l6WT3xsDPec04efjzyR9q3qv9IMNVs61jRbfe3QH17/Ia99/Zpn5wVyw7xGgcbgT02godqGomGrIzQW690f3M3U5VOpqKo45nHxLeJZd/c6Tx1LTbGvpIxn563n5c83UaXK9cNO4M5RfUhOiA/J67myO244WeIIv9rdHG8YeAMXzLjAs0VNNxRlmxpDY11O6yuod2nbhfTE9LB2U20o1vriHNR1ECsmrAh5bE4pKavgpc82MfXT9RworeCyU1O4/7y+pHZqE9LXDSRxWFOVCYraM63e/cHdIZ/dNZTCMTttqGJobNbb+pqIrup/Vdg/m4ZirS/OEWkjwhVeWJVVVPHW0i088VEehQdKOe+krjw0JpPMbglOh3Ycmx3XhITXC+ZuiD9UMbhxHqy6eCXO5qqqUt79ahvnPfopv//3t5yY1JZ/3nEGz9+c5cqkAZY4TIh4fV4gN8TvbwyBDpDzyky3XomzqVSVj9fsZNyTC7jvrRW0jY/lpVtO463bT2foCZ2cDq9BVuMwIeGGGkFzuCF+GyAXuZZu2kvOzDUs3fQ9aZ3a8MDovlw0sAcxMcEb0BcoK45b4nAFr88L5Ib4G4shGueb8rI1O/YzaWYuH63ZRXJCPPecm8HVWam0jHW+8ccShyUOEyXq63mU3SObxbctdiAiU5cte0p4bO5a3l2xjXbxsdxxdm9+emY6bVoGoX9S+/ZQXHz89oQE2L/f76exXlXGRImgDL4M0onHHG93cSlTPl7H60u2ECPC+JEncsdZvenQpmXwXqSuz66h7UFgicMYD6s95UiTeh45cOKJdPsPlzP90w28sHAjZZVVXH1aKveck0G3xFZOhxYUljiM8bDaU454sZYUSQ6XV/Lqok08M289+0rKuXBgdx4YnUmvpLZOhxZUljiMCYFwTlfS2IA/E3oVlVX8Y/lWHp+7jh37D3NW32QeGpPJgJREp0MLCUscxgRZ7S6yK3asYMaqGdZFNgKpKh9+s4NHZuWyofAgp6Z14LGrB3NG785OhxZSljiMCTI3TFdiQktVWZhXSM7MXFZtKyKjSzum3TiU0f27BnVxJb8kJNTfuSFELHEYE2RumK4kIA6ceMIqyL3GVuTvI2fmGj5fv4eUDq155MpBXHZqCi2cGrznQM83SxzGBJnn1ieJ9C63Qeo1lrermEdmrWXmtzvo3LYl/3Vhf64/PY342BZBCNJbHBuuKCKZIrKixs9+Ebmv1j5ni0hRjX3+y6l4jfFXtEzO15hA59Byq237DvHQ31cy+rH5LMwr5P7z+vLpxFHc+qNeUZk0wCUjx0WkBbANGKaqm2tsPxt4UFUvDOT5bOS4cZobpitxkqvm0Gqo5tDA+W/vwTKe/iSPvy3aDAI3nX4CvxjVh05tgzh4z0W8OHL8XGB9zaRhjJdFexdZL3cQOFBawQsLNvLcgg2UlFVwxdCe3HteX1I6tHY6NNdwS+K4BnijnvvOEJGVQAHVVx/fhi8sY0xTeK6DAFBaUcnri7cw5eM89hwsY+zJ3XhwTF/6dImQTgJB5HjiEJGWwMXAb+q4+0vgBFU9ICLjgHeBjHqeZzwwHiAtLS1E0RqnuWEdcNM4V3UQaKTXWGWV8s5X23hszlq27TvEGSd25lcX9GNwaocwB+odjtc4ROQS4E5VHe3HvpuALFUtbGg/q3FEJle1m3tNmCcy9MJnparM+W4nk2blsm7XAU5JSWTi2Ex+1Ccp/GMxXMBrNY5rqaeZSkS6ATtVVUUkm+peYHvCGZxxDy+3mzsuzBMZun0OrUXr95Azaw1fbdlHr6S2PH3dEC4Y0M3RhZS8xNHEISJtgPOB22tsmwCgqlOBK4A7RKQCOARco05fInlJhE2X7cV286Bzy2fqRxxu7CDwzbYicmblMn/tbrq1b8VfLz+FK4b2JLaF8wspeYmjiUNVS4DOtbZNrfH7FGBKuOOKGBE2Xbar2s2d4pbP1C1x+Glj4UEmz87l/a+3k9g6jt+O68dNZ6TTKi46x2E0lxuaqozxS1DWnjBRZef+wzzx0TreWppPyxYx3DmqN+NH9iaxdZzToXmaJQ7jGeFsN7feW95WVFLOs5+u5+XPN1JZpVw/LI27zulDl4TIWEjJaY73qgoF61Xl08QRs9HO1T2CmvqZBrs24tJj61BZJS99vpGp89ZTXFrBJYN68MvzM0nr3MaxmI5yS32qHl7rVWWMq0Rk7y0XnJhCqbyyireW5vPkR+vYVVzKuf268OCYTE7q3t7p0H7gsbpQQyxxRLJIny47RFzde8stn6lL4qiqUv7v6wIenbOWzXtKOC29I89cP4Ss9E5hjSPaWOKIZBH+LTNUXNF7q6FmDTc0Mzp8bKkq89buJmdmLqu376dftwRe/GkWozK7ROXgvXCzxGFcwy0FaVf03oqEZo0Qtekv37yXh2fmsmTjXtI6teGJawZz0cAeNngvjKw4blzBbQVpx6dFd2nxOSBBfg+5O4qZNCuXuat3ktQunnvO7cM1p6XRMtYjg/dc/placdx4jtsK0m4c9dwgl/fYaY78vSU8Nnct73y1jXbxsTw0JpNbhqfTpqXHTl8uqQsFg8f+8iZSubog7QWR0LRVy+7iUp7+JI8ZizcTI8L4kSdyx1m96dDGowspeTyB12SJw7iCKwrSgYjgb/hB0b7p3WD3Hy7n+fkbeH7hRkorqrgqK5V7z82gW6IN3nMLSxzGFVxRkA5EqL/he71Zowl/h8Pllfxt0WaemZfH9yXlXDiwO788vy8nJrcLQYCmOSxxGFdw+zTcYRfJVy21kl9FZRX//HIrj89dx/aiw4zISGLimH6c0jPRoQBNYxpMHCLSHkhW1fW1tg9U1a9DGpmJOp4rSJum8SVFVWXmNzuYNDuXDbsPMji1A5OvGsSZvZOC8zrWnBgy9SYOEbkKeBzYJSJxwE9Vdanv7peBIaEPzxjjF481bX2WV8jDM9fw9dYi+nRpx7QbhzK6f9fgDt6LwA4DbtHQFcdvgaGqut23+t7fROS3qvovwEbaGOMmHvkGvbJbBpOeX8zCvEJSOrRm0hUDuXxIT1rY4D1PaShxxKrqdgBVXSIio4D3RaQn4PxoFWOc5LFv+GFX6++T16knk0feyIeZw+m0fT+/v7A/1w9Ls4WUPKqhxLFfRHofqW/4rjzOBt4FTg5WACKyCSgGKoGK2iMXpfra9QlgHFBCdZPZl8F6fWOaxCPf8B3j+/sU7DvEE3PX8ffl+bSOa8E9I07kthG9SGjVxIWUwlG3sNpIoxpKHL+iVpOUqhaLyFjgN0GOY5SqFtZz3wVAhu9nGPCs719jjEt9f7CMZ+bl8cqizaDw0zN7ceeo3nRuF1+9Q1NPzuGoW1htpFENJY5XgGkiMllVKwBEpCswGcgE/hSG+AAuAV7V6km1vhCRDiLS/UgzmjHGPQ6WVvDiwo1Mn7+Bg2UVXHZqT+4/P4OeHWstpBSOk7M1J4ZMQ4ljKPC/wFcici9wCvBLIAe4KYgxKDBbRBSYpqrTa92fAuTXuL3Vt+2YxCEi44HxAGlpaUEMz5gmiqImj7KKKt5YsoWnPl5H4YEyRvfvyoNjMunb1cGTdIT9jd2k3sShqt8DE3xJYy5QAJyuqluDHMNwVS0QkS7AHBFZo6rza9xfV3eL44rzvoQzHapnxw1yjMYpXj75RkGTR2WV8u8V23h0zlq2fn+I00/sxPSb+jEkraPToZkQamgcRwfgYarrCWOpLk5/KCL3qurHwQpAVQt8/+4SkXeAbKBm4tgK1Bw+3JPqJGaiQRScfL1IVflo9S4mzcold2cxJ/doz18uO4WRGUm2kFIUaKip6kvgGeBOX41jtogMBp4Rkc2qem1zX1xE2gIxvqJ7W2A0x9dO3gPuEpE3qU5iRVbfMMY5izfsIWdWLss3f0+vpLZMue5Uxg3oHp6FlMJRt7DaSKMaShwjazdLqeoK4EwRuS1Ir98VeMf3DSUWeF1VZ4rIBN/rTQU+oPpqJ4/q7ri3BOm1janm5eawMPq2oIhJs3KZl7ubru3j+ctlA7gqK5W4Fk1YSKmpJ+dwfB72mTfKVgA07haOVdNC9RouX/HNX5v3HGTy7LW8t7KAxNZx/OLs3tx8ZroN3oswtgKgMW7g8SaPXfsP8+TH63hzST6xLYQ7R/Vm/MjeJLZu4uA9EzEscRh38/LJ16NNHkWHypn26Xpe/GwjFZXKtdlp3H1OH7q0t4WUTDVLHMbdPHry9WLd5FBZJS9/vompn65n/+FyLhnUg/vP78sJnds6HZpxGUscxoSCh7oRl1dW8fayfJ6Yu45dxaWMykzmoTH96N+j6cu/mshmicOYKFVVpby/ajuPzs5l054Ssk7oyJTrhpDdq5PToRmXs8RhTJRRVT5du5ucmbl8t30//bol8MLNWZzTr4sN3jN+scRhTBRZvvl7cmauYfHGvaR2as3jVw/mokE9bCElExBLHMZEolrF+bVJaUwacSNz+p5BUruW/PfFJ3NtdhotY5sweM9EPUscxoSC092Ifa+9tX0yj/3oev414BzalR3igfl/49ZPXqNtvP3XN01nR48xwTrJu6gLbmGbRJ4+4ypmDB4HKLcteYc7vvgHHQ8XQ/ybYY3FRB5LHMYE66Tugi64xYfLeX7BRp4f/xyH4uK5ctVc7v3sDXoU17fApgNclGCPcmNMLmaJw5gIcLi8khmLt/D0J3nsPVjGuI1f8ssFr9Fnb7CXzwkCFyRYv1/bheNu3MAShzEeVlFZxb++2sbjc9ZSUHSYERlJPDQmk4GpFzodmolgljiMcVoTmklUlVnf7uSR2bnk7TrAoJ6JTLpyEMP7JP3wWK/O8WVczxKHMU4LsJnk8/WFPDwzl5X5++id3JapNwxhzMndjh28Z+3yJoQscZi6WbEwcKH4ll8jGazq2pucc3/GgtSB9EhsRc5PBnL5kBRim7KQUnPYsRH1HEscIpIKvAp0A6qA6ar6RK19zgb+DWz0bfqXqtZeWtaEghULAxeik+b6Tik8OuIG/tNvBB1Livh/Pz6JG04/wbmFlJp7bLixGc2NMbmYk1ccFcADqvqliCQAy0Vkjqp+V2u/BapqlT4TdXa068wTw6/l7YHnE19Rxj2fvc5tS94h4cmS0L94Q1cVzeXGqxI3xuRijiUOVd0ObPf9Xiwiq4EUoHbiMCaqfN8qgamnX8HLQy6kKiaGG7/8D3cteoukkqLwBWFXnKYBrqhxiEg6cCqwuI67zxCRlUAB8KCqfhvG0IwJPV8zycG4VryUdTHThv2EAy1bc/k3H3PfwtdJ3b/L6QiNOYbjiUNE2gH/BO5T1drXi18CJ6jqAREZB7wLZNTzPOOB8QBpaWkhjNiY4Crbu483l27hyY/yKDxQyvn9u/LgA1eQWbjZ6dCMqZOoqnMvLhIHvA/MUtVH/dh/E5Clqg3On5CVlaXLli0LTpDRynrOhFxVlfLeygImz8klf+8hhvXqxMSx/Rh6Qkfn//4NrcvRUCG5ubE5/b6jmIgsV9Usf/Z1sleVAC8Aq+tLGiLSDdipqioi2UAMsCeMYUYv+08aMqrKx2t2MWlWLmt2FNO/e3teufUURmYk/TAWw81//1DGZrUVT3CyqWo4cCOwSkRW+Lb9FkgDUNWpwBXAHSJSARwCrlEnL5GMaaalm/by8IdrWLb5e9I7t+Gpa0/lx6d0JybcCyk19s3euqeaBjjZq2oh0OD/FlWdAkwJT0TGhM7q7fuZNCuXj9fsIjkhnv+5dABXn5ZKXLgH7x3R2Dd7N1/xGMc5Xhw3JpJt2VPCo3Ny+ffKAhLiY5k4NpOfnplOm5b2X894lx29JvK4oMC6q/gwT32UxxtLthDbQphwVm8mjOxNYpu4sLy+MaFkicNEHgcLrEWHypk+fz0vLtxEeWUV12Sncs85GXRp3yrkr+2oYCVrq614giUOY4LgcHklr3y+iWfmrafoUDkXDerBA+f3JT2prdOhhUewkrXVVjzBEocxzVBRWcXfl2/libnr2LH/MGdnJvPg6EwGpCQ6HVrD7Ju9aQZLHMY0pJ4mmKqE9ny4cA2TZ+eyofAgQ9I68Pg1gzn9xM4OBNkE9s3eNIMlDmMaUitpKLAg/VQmjbyJVa9/SWbXBJ67KYvzTupy7EJKxkQwSxwm8oSoGear7n3JOetmFp0wiJ77djD5ykFcemoKLcI9eM8Yh1niMJEnyM0w6zqn8siIG5mVeSadD+7jj3Omcu3KmcRPLQ/q6/jNBd2N63xtq5lEDUscxtRj275DPH7BvfxzwDm0KS/llwte49Zl/6Zd2SFnA3PjfE5WM4kqljiMqWXPgVKembeevy3aDP3P4tZl7/GLL/5Op0N2cjQGLHEYc9SB0gqeX7CB5xdspKSsgiuHpnLvPRfTY3sd62JYE4yJYpY4TNQrrahkxhdbmPJJHnsPlnHBgG48MLovfbokwBWbnA4vsrixPmMCZonDeEMITjiVVcq/vtzK43PXsW3fIYb36cxDY/oxOLVDM4P1Q7SeQN1YnzEBs8RhvCGIJxxVZfZ3O3lkVi7rdh1gYM9EHv7JQH6UkdTMIAPQnPdjPZiMwyxxmKiyaP0eHp65hhX5+zgxqS3PXD+ECwZ089bgvUi+IjGeYInDHCtCm1C+2VZEzqxc5q/dTbf2rXj4J6fwkyE9iXVqISVjPMzRxCEiY4EngBbA86r611r3xwOvAkOpXmv8alXdFO44o0qEtUFvLDzI5Nm5vP/1djq0ieN3407ixjNOoFVcC6dDC1yEJnXjPY7OLh+bAAAOMUlEQVQlDhFpATwNnA9sBZaKyHuq+l2N3X4GfK+qfUTkGuBh4OrwR2uCJkwnvx1Fh3nio3W8vSyfli1iuPucPtw28kTat/LwQkqRkNStPhMRnLziyAbyVHUDgIi8CVwC1EwclwB/9P3+D2CKiIiqajgDNUHU1JOfnyecfSVlPPvpel7+bBNVqlw/LI27z8kgOSG+iQGHSLSeQO3KKCI4mThSgPwat7cCw+rbR1UrRKQI6AwUhiVC4x6NnHAOlVXy0ucbmTpvPcWlFVw2OIX7z+9Laqc2YQowQHYCNR7mZOKoqxtL7SsJf/ap3lFkPDAeIC0trXmRGc8or6zizaX5PPnROnYXl3LeSV14cEwm/bq1dzo0YyKWk4ljK5Ba43ZPoKCefbaKSCyQCOyt68lUdTowHSArK8uasprKI00oVVXK/31dwKNz1rJ5TwmnpXfk2euHkJXeyenQjIl4TiaOpUCGiPQCtgHXANfV2uc94GZgEXAF8LHVN0LM5U0oqsq83N3kzMpl9fb99OuWwEs/PY2zM5O9NRajKTyS1E3kcyxx+GoWdwGzqO6O+6KqfisifwKWqep7wAvA30Qkj+orjWucitc0U329qY7w4+S3bNNecmbmsmTTXtI6teGJawZz0cAexETLQkouT+omekgkfoHPysrSZcuWOR2Gqamhq4FGjsE1O/bzyKxc5q7eRXJCPPecm8HVWam0jI2ywXs2jsOEkIgsV9Usf/a1kePGtbbsKeGxuWt5d8U22sXH8tCYTG4Znk6bllF62EbCOA4TEaL0f6Bxs93FpUz5eB2vL9lCjAi3j+zNhLNOpEOblk6HZozBEodxkf2Hy3lu/gZeWLiR0ooqrspK5d5zM+iW2Mrp0IwxNVjiMI47XF7Jq4s28cy89ewrKefCgd15YHQmvZLaOh2aMaYOljhMeNTRlbRCYvjHaRfy+KR57Nh/mJF9k5k4JpMBKYkOBWmM8YclDhMeNXr9qCoffrODR2bnsmH3QU7t0IrHrh7MGb07OxigB9g4DuMSljhMWC1cV8jDM9ewalsRGV3aMf3GoZzfv2vkD94LButya1zCEocJi5X5+8iZtYbP8vaQ0qE1j1w5iMtOTaGFWwbv2RgJY/xmicOEVN6uYh6ZtZaZ3+6gc9uW/OGi/lw3LI34WJctpGRjJIzxmyUOsG+bIVCw7xCPz13LP5ZvpU3LWO47L4OfjziRdvFRdsjZsWUiUJT9L66HfdsMmr0Hy3jmkzxe/WIzKNwyvBe/OLs3ndu5bCGlmtqHcAp2O7ZMBLLEYYLiYGkFLyzcyPT5Gygpq+DyIT25//y+pHRo7XRojbOTuDEBscRhmqW0opI3Fm9hyid5FB4oY8zJXXlwdCYZXa2LqDGRyhKHaZLKKuXfK7bx6Jy1bP3+EGec2Jnnbsrk1LSOTocWXDZGwpjjWOIwAVFV5q7exaRZa1i78wADUtrzv5efwo/6JEXuWIy63pcVt00Us8QBNiLXT4s37OHhmWv4css+eiW1Zcp1pzJuQPfIXkipucVtO7ZMBLLEAfbNsRHfFhQxaVYu83J307V9PP97+SlcMbQncS0iZCGlhk7uzS2c27FlIpAjiUNEJgEXAWXAeuAWVd1Xx36bgGKgEqjwd3UqExybCg8yec5a/m9lAYmt4/jNBf24+cx0WsU5MHgvlOMhGnp8pDa/GdMMTl1xzAF+41t3/GHgN8Cv6tl3lKoWhi80s3P/YZ78aB1vLc0nrkUMd47qzfiRvUlsHedcUDYewhjXcCRxqOrsGje/AK5wIg5zrKKScqbOX89Ln22kolK5blgad53Thy4JtpCSMeYHbqhx3Aq8Vc99CswWEQWmqer08IUVPQ6VVfLy55t4dl4exaUVXDo4hfvP60ta5zZOh+Y8K24bc5yQJQ4RmQt0q+Ou36nqv337/A6oAGbU8zTDVbVARLoAc0RkjarOr+f1xgPjAdLS0podfzQor6ziraX5PPnROnYVl3Juvy48OCaTk7qHcAoOr7HitjHHCVniUNXzGrpfRG4GLgTOVVWt5zkKfP/uEpF3gGygzsThuxqZDpCVlVXn85lqVVXK+6u28+jsXDbtKeG09I48ff0QTkvv5HRoxhgPcKpX1Viqi+FnqWpJPfu0BWJUtdj3+2jgT2EMM+KoKp+u3U3OzFy+276fft0SeOmnp3F2ZrL7B+9Zk5ExruFUjWMKEE918xPAF6o6QUR6AM+r6jigK/CO7/5Y4HVVnelQvJ63fPNeHp6Zy5KNe0nr1IYnrhnMRQN7eGfwnjUZGeMaTvWq6lPP9gJgnO/3DcCgcMYViXJ3FDNpVi5zV+8kqV08f7rkZK45LY2WsREyeK8+tg6GMSHjhl5VJgTy95bw2Ny1vPPVNtq1jOWhMZncMjydNi2j5CO3cR/GhEyUnEWix+7iUp7+JI8ZizcTI8JtI07kjrN607FtS6dDM8ZECEscEaL4cDnPzd/A8ws3UlpRxVVZPbnn3Ay6J3pgISVjjKdY4vC4w+WVvPbFZp7+JI/vS8oZd0o3HhidSe/kdk6HZoyJUJY4PKqisop/fbmNx+eupaDoMCMyknhoTCYDe3ZwOjRjTISzxOExqsqsb3cwaVYu63cfZFBqBx65chBn9klyOjR3sXEfxoSMJQ4P+TyvkIdn5bIyfx99urRj6g1DGHNyN/cP3nOCdbk1JmQscXjA11v3MWlWLgvWFdIjsRU5Vwzk8lNTiI2UhZSMMZ5iicPF1u8+wOTZuXywagcd28Tx/358EjecfoIzCykZY4yPJQ4X2l50iCfmruPvy7fSKjaGe8/N4OcjepHQysGFlIwxxscSh4t8f7CMZ+bl8cqizaBw0xkncOeoPiS1i3c6NGOMOcoShwscLK3gxYUbmT5/AwfLKrh8SE/uOy+Dnh1tISVjjPtY4nDY53mF3PPmVxQeKGN0/648OCaTvl2ty6gxxr0scTisV3Jb+vdI5L7zMhiS1tHpcIwxplGWOBzWPbE1r96a7XQYxhjjNxsIYIwxJiCWOIwxxgTEkcQhIn8UkW0issL3M66e/caKSK6I5InIr8MdpzHGmOM5WeN4TFUfqe9OEWkBPA2cD2wFlorIe6r6XbgCNMYYczw3N1VlA3mqukFVy4A3gUscjskYY6Kek4njLhH5WkReFJG6+qGmAPk1bm/1bauTiIwXkWUismz37t3BjtUYY4xPyBKHiMwVkW/q+LkEeBboDQwGtgOT63qKOrZpfa+nqtNVNUtVs5KTk4PyHowxxhwvZDUOVT3Pn/1E5Dng/Tru2gqk1rjdEygIQmjGGGOawZHiuIh0V9XtvpuXAd/UsdtSIENEegHbgGuA6/x5/uXLlxeKyOagBBseSUCh00E0gcUdPl6MGSzucGpuzCf4u6NTvapyRGQw1U1Pm4DbAUSkB/C8qo5T1QoRuQuYBbQAXlTVb/15clX1VFuViCxT1Syn4wiUxR0+XowZLO5wCmfMjiQOVb2xnu0FwLgatz8APghXXMYYYxrn5u64xhhjXMgShztMdzqAJrK4w8eLMYPFHU5hi1lU6+3haowxxhzHrjiMMcYExBKHS4jIn30j6VeIyGxfDzPXE5FJIrLGF/s7ItLB6ZgaIyJXisi3IlIlIq7vOePFyT59M0LsEpG6utq7koikisgnIrLad3zc63RM/hCRViKyRERW+uL+75C/pjVVuYOItFfV/b7f7wH6q+oEh8NqlIiMBj72dZ9+GEBVf+VwWA0SkZOAKmAa8KCqLnM4pHr5JvtcS43JPoFr3T7Zp4iMBA4Ar6rqAKfj8YeIdAe6q+qXIpIALAcu9cDfWoC2qnpAROKAhcC9qvpFqF7Trjhc4kjS8GlLA9OruImqzlbVCt/NL6ge4e9qqrpaVXOdjsNPnpzsU1XnA3udjiMQqrpdVb/0/V4MrKaB+fHcQqsd8N2M8/2E9PxhicNFROQvIpIPXA/8l9PxNMGtwIdOBxFhAprs0wSHiKQDpwKLnY3EPyLSQkRWALuAOaoa0rgtcYRRIxM/oqq/U9VUYAZwl7PR/qCxuH37/A6ooDp2x/kTs0cENNmnaT4RaQf8E7ivVkuAa6lqpaoOpvqKP1tEQto86ORCTlHH34kfgdeB/wB/CGE4fmssbhG5GbgQOFddUjQL4G/tdjbZZxj5agT/BGao6r+cjidQqrpPROYBY6l7DsCgsCsOlxCRjBo3LwbWOBVLIERkLPAr4GJVLXE6ngh0dLJPEWlJ9WSf7zkcU0TyFZlfAFar6qNOx+MvEUk+0ptRRFoD5xHi84f1qnIJEfknkEl1b5/NwARV3eZsVI0TkTwgHtjj2/SF23uDichlwFNAMrAPWKGqY5yNqn4iMg54nB8m+/yLwyE1SkTeAM6mesbWncAfVPUFR4NqhIj8CFgArKL6/yHAb31z5rmWiAwEXqH6+IgB3lbVP4X0NS1xGGOMCYQ1VRljjAmIJQ5jjDEBscRhjDEmIJY4jDHGBMQShzHGmIBY4jAmBHwzrW4UkU6+2x19t08QkZkisk9E3nc6TmOawhKHMSGgqvnAs8BffZv+CkxX1c3AJOBGp2IzprkscRgTOo8Bp4vIfcCPgMkAqvoRUOxkYMY0h81VZUyIqGq5iDwEzARG+6ZFN8bz7IrDmNC6ANgOeGIxI2P8YYnDmBARkcFUr9x3OnC/b4U5YzzPEocxIeCbafVZqtd02EJ1QfwRZ6MyJjgscRgTGrcBW1R1ju/2M0A/ETlLRBYAfwfOFZGtIuLamXmNqYvNjmuMMSYgdsVhjDEmIJY4jDHGBMQShzHGmIBY4jDGGBMQSxzGGGMCYonDGGNMQCxxGGOMCYglDmOMMQH5/9s9mn9jubgvAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from numpy import *\n",
    "dataArr,labelMat = loadDataSet()\n",
    "weights = stocGradAscent0(array(dataArr), labelMat)\n",
    "plotBestFit(weights) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4-4 改进的随机梯度上升算法"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "def stocGradAscent1(dataMatrix, classLabels, numIter = 150):\n",
    "    m,n = shape(dataMatrix)   # 取数组（narray）的行，列 m=100， n=3  （narray 和 matrix 长的一样）\n",
    "    weights = ones(n)  # [1. 1. 1.]\n",
    "    for j in range(numIter):  # 循环到 最大循环次数numIter = 150：\n",
    "        # dataIndex = range(m) ### 书中需要更改的地方  tip: TypeError: 'range' object doesn't support item deletion\n",
    "        dataIndex = list(range(m))  # 创建一个0到99， 步长为1 的列表\n",
    "        for i in range(m):   # 循环listIndex\n",
    "            alpha = 4/(1.0+j+i)+0.01\n",
    "            randIndex = int(random.uniform(0, len(dataIndex)))   # 随机整数值，返回 0 到 m - 1（这里是99）\n",
    "            h = sigmoid(sum(dataMatrix[randIndex]*weights))   #  1.0/(1+exp(-Z))    Z = dataMatrix * weights\n",
    "            error = classLabels[randIndex] - h\n",
    "            weights = weights + alpha * error * dataMatrix[randIndex]   # 更新权重\n",
    "            del(dataIndex[randIndex])   # 删除用过的 随机数 （避免重复）\n",
    "    return weights   # 返回权重"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY4AAAEKCAYAAAAFJbKyAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3X+UXGWd5/H3t7vzo5N0dxrSQNLdGEUEEQkjPR1cRoVFMWRccfbMzIGzKDs6JPFoRnfWjToedVbHORp19Cz+gDCgqIizsyqyGhT8MQdlIaSDCYEJKKLQnQAJJOkk5Genv/tHVYfq6qrqut116z731ud1Tp+uunW78q2uzv3W8zzf53nM3REREalWU9IBiIhIuihxiIhIJEocIiISiRKHiIhEosQhIiKRKHGIiEgkShwiIhKJEoeIiESixCEiIpG0JB1AHBYsWOCLFy9OOgwRkdTYtGnTc+7eVc25mUwcixcvZmBgIOkwRERSw8yerPZcdVWJiEgkShwiIhKJEoeIiEQSe+Iws5vNbKeZPVxw7O/NbLuZbc5/LS/zs8vM7DEze9zMPhR3rCIiMrl6tDi+DiwrcfwL7n5+/mt98YNm1gx8GbgcOAe4yszOiTVSERGZVOyJw93vAXZP4Uf7gcfd/Ql3Pwp8B7iipsGJiEhkSY5xvNfMHsp3ZXWWeLwbGCy4P5Q/JiIiCUoqcXwVOAM4H3ga+HyJc6zEsbL73JrZCjMbMLOBXbt21SZKkRQbHB5k9frV9N/Yz+r1qxkcHpz8h0SqkMgEQHd/duy2md0I/LDEaUNAb8H9HmBHhedcB6wD6Ovr00bq0tAGhwdZcv0SDhw9wLHRY2x+ZjO3br2VLau20NvRO/kTiFSQSIvDzBYW3P0z4OESp20EzjSzl5rZTOBK4I56xCeSdmvvXXsiaQAcGz3GgaMHWHvv2oQjkyyIvcVhZrcBFwMLzGwI+DhwsZmdT67r6Q/Ayvy5i4B/dvfl7j5iZu8FfgI0Aze7+yNxxyuSBRu2bziRNMYcGz3GA9sfSCgiyZLYE4e7X1Xi8E1lzt0BLC+4vx6YUKorIpUt7V7K5mc2j0seM5pm0N/dn2BUkhWaOS6SQWsuWsO8mfOY0TQDyCWNeTPnseaiNQlHJlmgxCGSQb0dvWxZtYWVF6ykf1E/Ky9YqYFxqZlMLqsukhWDw4OsvXctG7ZvYGn3UtZctKbqi39vRy/XLb8u5gilESlxiARKJbUSKnVViQRKJbUSKiUOkUCppFZCpcQhEpDCZUIOjxymxcb3JodSUqvlTBqbxjhEAlE8ptHS1MJxP06LtTDiI8GU1GrsRdTiEAlE8ZjGyOgILU0tvLLrlUGV1GrsRdTiEAlEuTGN1pZWNly7IaGoJtLYi6jFIRKIpd1LT8z0HhPKmEahtMQp8VHiEAlEWpYJSUucEh8lDpFApGWZkHrEqaqtsJl79vY86uvr84GBgaTDEJEpKK7aGmvRhJhEs8TMNrl7XzXnqsUhIkFR1Vb4lDhEJCiq2gqfEoeIBEVVW+FT4hCRkpIaoI6zakuD7rWhwXERmSDpAeqxfUh++dQvGfVRmqyJ153+ukj7kZR6Tg26lxfU4LiZ3WxmO83s4YJjnzWzR83sITP7vpnNL/OzfzCzrWa22cyUCUTqJOkB6t6OXtZctIanhp/i0eceZcuzW7hh0w0suX7JlFsJSb+mLKlHV9XXgWVFx+4GznX384DfAB+u8POXuPv51WZCERlvKt0zIQxQ1/pCH8JryorYE4e73wPsLjp2l7uP5O/eD/TEHYdIIxrrnrlh0w1s3LGx6k/tIQxQ1/pCH8JryooQBsffCdxZ5jEH7jKzTWa2otKTmNkKMxsws4Fdu3bVPEiRNProzz/K3sN7I39qD2FZkVpf6EN4TVmRaOIws48AI8CtZU65yN1fA1wOvMfMXl/uudx9nbv3uXtfV1dXDNFK3FTxUluDw4N8a+u3cMYXwFTzqT2E5U9qfaEP4TVlRV2qqsxsMfBDdz+34Ng1wCrgUnc/WMVz/D1wwN0/N9m5qqpKH1W81N7q9av5ysavMMrouOOG8Y7z3kHbrDY2bN/A0u6l06pWitNYddUD2x+gv7s/2DizIEpVVSL7cZjZMuCDwBvKJQ0zmws0ufv+/O3LgE/UMUypo0oDodctvy7h6NJpw/YNE5IG5BLH7Y/dzsFjB4Pfwa+3o1fvf4DqUY57G3AfcJaZDZnZu4AvAW3A3flS2+vz5y4ys/X5Hz0V+JWZbQEeAH7k7j+OO15Jhipeaq/UGEETTZx58pknkgaoLFWii73F4e5XlTh8U5lzdwDL87efAJbEGJoEZGn3UjY/s3lc8gi54mWsCyXkrp41F63h1q23Tuj+m90yW0lapiWEqiqRVFW8TLXEtd7KDQa/7vTXqSxVpkVLjkgw0jIQunr9am7YdMOE1tHKC1amoj8+pEKE4pbb1eddzbce+lbQLbmsijI4rsQhDWdweJCP/vyj3Pn4nWBw+csv55OXfLLqC1T/jf1s3LFx4vFF/Wy4dkOtw41FCEm6OIG1WAvH/TjNTc2MjI6osq7Ogq+qEknK4PAgr/7qqxk+Mnzi2C1bbuH2R29n67u3VnWBStt4TCkhVCsVV9KN5BeTGBnNfVdlXbg0xiGZUO3kwbX3rmXfkX0Tju8/sr/qqqI0jceErFQlXTEN2odJLQ5JveIuj0rzEjZs3zBhJjXAKKNVX6DGBp2T7upJu1Itt2Jpa8k1CrU4JPWirKK6tHsphk043kRTpAvUWFfPhms3cN3y65Q0pqC45dZiLRhGS1Pu86xacuFS4pDUizJ5cM1Fa2if1T7heNustoa4QIW0HlhxufCqvlXc9677WHXBKq0lFTh1VUnqRRms7u3oZeu7t1asqgp5ct90Yivu0vv107/mxgdv5OwFZ097d72pKjVIv7RnaV1jkOhUjiupV8t5CSHNcah1bKXmn4wJ6XVKMoLaOlYkbrVcLjvk7UWnG1ulKqaQXqeET11Vkgm1mpcQx2KLter6mm5sk1UxTfZcIXfhSX0pcYgUqPXkviilwnHHVrzoYbFKz1XL1yHpp64qSbVaVwnVenJfLbu+phtbYZfeklOXMKt5VtWlryF34Un9qcUhqRXHp+BaT+6bTvdSqa6h6cZW2KUXZb0q7ZcihZQ4JLXi2jWw0nhJ1H7+c7rOYdOOTeN24qume6lSUqzVuk1RxoWysD6X1I66qiS16v0pOOo+HIPDg9z+6O0Ttm+dM2POpN1LoXUNZWV9rpAmQKaZEoekVqmtUeP8FBz1Yr723rUcPHZw3DHDeNtZb5u0eym0rqFaljwnJS0bcKVBXRKHmd1sZjvN7OGCYyeZ2d1m9tv8984yP3tN/pzfmtk19YhX0qHen4KjXsxLne84257bNum/Ve+kWI20r88VWisuzerV4vg6sKzo2IeAn7n7mcDP8vfHMbOTgI8DS4F+4OPlEow0nnp/Co56MZ/OxT8rXUMhCa0Vl2Z1SRzufg+wu+jwFcAt+du3AG8r8aNvBu52993uvge4m4kJSBpYPT8FR72YT+fiX6+k2Eh9/iG24tKqbmtVmdli4Ifufm7+/l53n1/w+B537yz6mQ8As939H/L3PwoccvfPVfq3tFaVxCXqlqshbNFaTsjrcsWh0V5vVFnaOnbixgmU2IUHMLMVwAqA008/Pc6YpIFFXdqkHlu0TnUpkLjKmUOlDbhqJ8nE8ayZLXT3p81sIbCzxDlDwMUF93uAfyv1ZO6+DlgHuRZHbUMVCdN0JkGG0OdfTdKr5RpZIey1ngVJJo47gGuAT+e//6DEOT8B/rFgQPwy4MP1CU+i0AJ4ySjXavjTb/8ps1tmV3wvyi16eGjkEIPDg7G/f9UkPa2RFaZ6lePeBtwHnGVmQ2b2LnIJ401m9lvgTfn7mFmfmf0zgLvvBj4JbMx/fSJ/TAKi+vjklGs1bN25ddL3Ymzwfmy9qjHbdm2ry/tXTXmsSmjDVK+qqqvcfaG7z3D3Hne/yd2fd/dL3f3M/Pfd+XMH3P2vC372Znd/ef7ra/WIV6LRf+7klKoUKlTuvRhrIfZ29NI+c/xWuiM+Upf3r5qusnueuifx7jSZKPTBcUmBEPrKG9VkS6XDxPeiuPunmp+Jw2TrXw0OD/LYc49N+LkWa2n4EtoDR0YYdad9dvkPDXFS4pBp0wJ4ySmuFDo0cohtu7Yx4iMnzil+L4pbiKXU4/0rTnrF81zW3ruW4358ws81NzVnfiLk8KFjDO05yPY9hxjac4jtew/l7u/N3d978Bh/+6ZX8DeXnplIfEocMm3lLgBXn3c1q9evTu2AeQgD/tXEULxUeqm5CoUX2kpbyEL9ZqlPVh67YfsGRkZHJvzc2QvOTtXfUTF3Z/cLR08kgVxyODju/v4j419364xmujtb6elsZUnPfHo65/DaM05O6BXUcQJgPWkCYP0VT3S7+ryrufzWy1M72SqEyWJTjWGySYer16/mhk03TGghnr3gbFpbWoOZ31AuzpUXrAy6pHZ01HnuwBEGC1sKJ27nEsOhY+NbUm2zWk4khu75rfR0zjlxv6dzDp1zZmBWalpb7USZAKjEIbFI63/6MSHEH1cMISTFaoQa5/FR55l9h19sKRQmhb25xHD0+Pil9DvnzKC7ICkUJ4iO1mTGKgplaea4pFTaB8xDiD+uGNIygzqpOI+OjPLM8GGG9hxkaO+hgnGGgwztOcQzw4cZGR3/gXvBvFl0d7ZyzqJ2LnvVqfQUJIXu+a3MnZWtS222Xo0EI+0D5iHEX20MUxmLScsM6jjiPHzs+ImWQWFX0liL4Zl9hynsiDGDU9tm09PZSt9LOvPJIN9qyCeG2TOaaxpj6NRVJbEItZuhWiHEX00MIcQZmheOjJxIDGOthqE9L44vPHfgyLjzm5uMhR2z6Z6fSwQ9Bd1JPZ1zOK1jNjNbsr/nncY4lDiCEPLKsNUIIf6pDnSnZSxpKoYPHStZiTTWethzcHz33szmJhbNn116jOGkOZzaNouW5uwnhskocShxSIPov7GfjTs2Tjy+qJ8N125IIKLpcXf2HCw1h+HFRLH/8PhS1dkzmvKthbFWQn58YX7udte8WTQ1xVuRlKj2dti/f+LxtjbYt6/qp9HguEiDqMlYTI0uPNUYK1UdGtdSOFgwn2Fiqeq8WS0nWghLX3rSuEHnns5WTpo7M/ZS1aCVeu8qHa8BJQ6RFJts9nVVanjhOT7qPLvv8IkqpOJWw/a9hzg6Mr5UtaN1Bj2drbysay6vf0XXiYSQG2+YQ3trS2MnhgApcYikWL1LVo8dz5WqDpZYDqOqUtVzTi2Y6JZrOczLWKlqI9A7JhKDei5XUsuS1cPNM9jR3sX2jlMY6jiV7T95bFzJ6jP7DjNaolS1u7OV15ze+WJLIT/G0D2/ldaZjVWq2gg0OC5SYyGXyB48OpJrKRSuk3T919jefgpDHaewa95J485vbjJOa5/94vIXRcthLOxobYhS1aBV6saLcH3X4LhIgpLcy3vf4WMvdiHlu48Kxxd2v3B03Pkzmo1Fp72cnuGdXPLEAD3DO+ke3knP8LN079vJabufUalq6Nrayhc3xESJQ6TG4loqZKxUtXgOw4trJB1kX4VS1Vf3dJwYeB4bYzilbRZN8zvKX3iUNMJX48q3aihxiNTYVEtk3Z1dB45M3IOhoNVw8Oj4UtW5M5tPTGj748Wd4ye4dbZycjWlqglceOqqjuXGjSKxMQ4zOwv4l4JDLwM+5u5fLDjnYuAHwO/zh77n7p+Y7Lk1xiFJKjfG8eCKzcxq6ipZjTSWHI6UKVU9sRxGwcS2nvyqqqGWqoawnwlQszGArEvdzHEzawa2A0vd/cmC4xcDH3D3t0R5PiUOScJYqerQnkM8tGOI7z78C57afYC5Taczyxaya/8xjh0f///t5Lkzx+27MFaJ1HNS7ntbQluDTldQBQJKHFVJ4+D4pcDvCpOGSGiOjBxnx97DJddJGtpzsESp6pmc0jaL7o7x3UeNUKqaZIGAxC+UxHElcFuZx15rZluAHeRaH4/ULyxpJIWlqtuLBp2H9hxi5/7xq6o2GSzsyCWDC884ecIeDAvnz2ZWSzYTw2RC2M9E4pN44jCzmcBbgQ+XePhB4CXufsDMlgO3AyV3ZzezFcAKgNNPPz2maCXN9h8+VnqP5/z3kqWq+ZbBxWd1jduDoaezldPaZ6tUtYwQ9jOR+CQ+xmFmVwDvcffLqjj3D0Cfuz9X6TyNcWRXuQFXd2fvwWPjB5yLupKKS1VntTRNWGq7cCD6lLbZNGd5VdUYBTXGoaqqqqRtjOMqynRTmdlpwLPu7mbWDzQBz9czOAmDu/PQ07/n0q/9JUePzsOO9/L7J0f53n238IqOC9m5b4QXypSqdne20pcvVS0cY1gwr4FWVa3zxTOo7WmVHGou0RaHmc0BBoGXuftw/tgqAHe/3szeC7wbGAEOAX/r7v9vsudViyN9jo86O/cfLl2qmh9zKC5VPc4BRpt20tM5l+VnLy0oU50TfKlq3dW6skif4jMndeW4tabEkRfQf+6R46M8PXx4QvfR2P2nhw+VLFUt3Nf5tm1f5ol9A4zYs4zYTtwOAendtGhKpvqe1jpxqMQ1c9LWVSVxqeMGL4WlqsUb82zfm0sMRatt50pVO1tZ0juf5a9eeCJJ9Ha2smh+K3Nmjv/zfMaa+M2mXzf2gGsCm/aIFFPikKocOnr8xYRQ1GqoWKqa37Wtu2j288KO2cyeEa1UtSabFonItClxCJArVd1eNH+hcN/n50uUqo4lhje8omvCchindcxmRo1LVYMacBVpYEocDcCB4dnzGGrPb87T0cXQ/31kXJIYPjR+stbMliZ68i2Eyxa1T1g8L6lS1VpuWlRJMOss1UoCS29LkYDGHKdLg+MZ4O48/8LRiYPO3/zX3E5u7afwwqw5435mzszmCYvnFd5fMHcWTQ06hyGoOQjFQhmUztBFsG5Cee/K0OB4xoyOOjv3HzkxxjBU1J20Y+8hDh8bX6raPruF7vmn0LvnGV775EP0DD9Lz778Jj3HDzJ/53aVqpYR9DpLobQclBwamhJHAApLVV/sPnpxIPrpvYc5enx8YhhbVfWsU9u49OxTTmzWM9aV1D57BvDmZF5QygW9zpIu2BIAJY6E/eLRnfz1NwY4XlSr2tU2i57OVl7d3cHl5y4ct+dzd+fEUlWpnSDWWcpCV1AWXoOUpKtPws7omse733DGicTQPT83hyFqqWoWhDIgHUTZbxbma2ThNUhJFQfHzawd6HL33xUdP8/dH4o7uKlqtMHxLAhtQHosiSVW9ht1IDXET/eBDwbXXYjvUYGaDI6b2V8CXwR2mtkM4L+6+8b8w18HXjPdQEXGhDYgXa+y35rRp/vwBZAcaqXSDK2/Ay5w9/OBvwK+aWb/Of+YynGkpoIekC6lvT33ibr4q7096cjCoN9DplUa42hx96cB3P0BM7sE+KGZ9ZCbUyZSM0EMSEehT/iV6feQaZVaHPvM7IyxO/kkcjFwBfCqmOOSBrPmojXMmzmPGU0zALQOVbl5GVmY6Z2F19DgKiWOD1LUJeXu+4FlwKfiDEoaz9g6VCsvWEn/on5WXrAyjJnaSdm3LzeAXPyVhX7yer0GdSfGpmxVlZk9AdwAfN7dR/LHTgU+D5zl7n9ctygjUlWVxC60iqHQKnZC+P2EEEOKRKmqqtTiuAB4KfBrM/uPZvY+4AHgPmDp9MMUkZrJcgtFglN2cNzd9wCr8gnjp8AO4EJ3H6pXcCLBCmXNqFDp95NpleZxzAc+Q651sQxYDtxpZu9z95/XKT6RMOmTfGX6/WRapa6qB4HfAn3ufpe7vx94O/APZnZbrQIwsz+Y2VYz22xmEwYmLOd/mdnjZvaQmWnioUijqseAtwbVJ1VpHsfri7ul3H0z8B/M7Noax3GJuz9X5rHLgTPzX0uBr6IxFpF0m+pgfpT5M1PtLtMcnUlVGuMoO5bh7jfGE05JVwDf8Fz51/1mNt/MFo5NThSRFKrHxVndZbGp7abQU+PAXWa2ycxWlHi8GxgsuD+UPzaOma0wswEzG9i1a1dMoYpEoC4PyagQEsdF7v4acl1S7zGz1xc9XqoYe0IRtruvc/c+d+/r6uqKI05JQpovvurykIxKPHG4+478953A94HixYmGgMLpwz3kSoOlEejiKxKcRBOHmc01s7ax28BlwMNFp90BvCNfXXUhMKzxDZEGVY81vLK8TliNJL0D4KnA9y23NEAL8G13/7GZrQJw9+uB9eTmkDwOHCS3xLtI7YS2XEcjmGrFUz3eD73nk0o0cbj7E8CSEsevL7jtwHvqGZc0GHWH1Z8uzqmW+BiHSGapy0MyKumuKpHK0rzmkT5VS0apxSFhS+uqr2kuIxaZhBKHSBw0biIZpsQhIiKRKHGIiEgkShwiIhKJEoeIiESixCESh6TncKiqS2KkeRwicUi6XFhVXRIjtThEatU60Kd8aRBKHCK1mmSoT/nVCTHBhhhTwJQ4RKS+QkywIcYUMCUOERGJRIlDJIuSruqSTFPiEElaHP3raV0cUlJBiUOkVqb6KV/965IyShxSmqpMoovjU36Iv//p/m2E2I0WYkwBSyxxmFmvmf3CzLaZ2SNm9r4S51xsZsNmtjn/9bEkYm1I+hQcpnr9/islh+n+bYTYjRZiTAFLcub4CPDf3f1BM2sDNpnZ3e7+70Xn/dLd35JAfCKNSx8cpILEWhzu/rS7P5i/vR/YBnQnFY+IiFQniDEOM1sM/BGwocTDrzWzLWZ2p5m9qq6BidSD+tElZRJPHGY2D/gu8H53L+5QfBB4ibsvAa4Dbq/wPCvMbMDMBnbt2hVfwCK1Vqp/vVGpKCMVEk0cZjaDXNK41d2/V/y4u+9z9wP52+uBGWa2oNRzufs6d+9z976urq5Y424IqjJJVsi//zhj09hKKiQ2OG5mBtwEbHP3fypzzmnAs+7uZtZPLtE9X8cwG5eqSZKV9O+/ra30xbqtLfnYJHFJVlVdBLwd2Gpmm/PH/g44HcDdrwf+HHi3mY0Ah4Ar3Ru5HS9SI+XKascSg5KDVJBY4nD3XwE2yTlfAr5Un4hEGoi6hGQaEh8cFxGRdFHikOxRZU56hVwUICcocUj2qBum/mqVrLX0RyoocYjI9ClZNxQlDpFGpC4hmYYky3FFJCnq+pFpUItDpBINtItMoMQh2VPLbhj13YtMoK4qyR51w9RfpSVKJHPU4hBJmxC7z1RG21CUOETSRt1nkjAlDhERiUSJQ6QSzXcQmUCJQ6QS9d3XVojjMxKZEoekQ9YuOFl7PdXS+EwmKHFIOmTtgjOd16PuM0mY5nGIpI26ySRhanHIeI3ahSIiVUs0cZjZMjN7zMweN7MPlXh8lpn9S/7xDWa2uP5RNpisdQlliZK6BCKxxGFmzcCXgcuBc4CrzOycotPeBexx95cDXwA+U98opeZ08Zu6LCR1jc9kQpItjn7gcXd/wt2PAt8Brig65wrglvzt/wNcamZWxxil1qZ68cvaBSdrr6daKm/OhCQHx7uBwYL7Q8DScue4+4iZDQMnA8/VJUIJR9YuLFl7PdJQkmxxlGo5+BTOyZ1otsLMBsxsYNeuXdMOTkRESksycQwBvQX3e4Ad5c4xsxagA9hd6sncfZ2797l7X1dXVwzhNohG7UIRkaolmTg2Amea2UvNbCZwJXBH0Tl3ANfkb/858HN3L9nikBpRH3S4lNQlEImNceTHLN4L/ARoBm5290fM7BPAgLvfAdwEfNPMHifX0rgyqXhlmtrbKw+A6+I3OSVvCUSiM8fdfT2wvujYxwpuHwb+ot5xSQwqJQ01IkVSRTPHRdJCc2AkEEocImmRhQmAkglKHCIiEokSh4iIRKLEIfWhUlKRzNB+HFIfKiUVyQy1OETSQq02CYRaHCJpoVabBEItDhHQHAmRCJQ4REBzJEQiUOIAfdqU+OhvSzJIiQP0abPRxXkR19+WZJASh4gu4iKRKHGIiEgkShwilWiOhMgEShwilWjuhMgEShygGblS3nSrovS3JRmkmeOgT5WNrq2t9AB5ueNQ/YC6/rYkg9TikHSIcz7Evn257WuLv3TRFykpkRaHmX0W+E/AUeB3wF+5+94S5/0B2A8cB0bcva+ecUpANB9CJBhJtTjuBs519/OA3wAfrnDuJe5+vpKGiEgYEkkc7n6Xu4/k794P9CQRh4iIRBfCGMc7gTvLPObAXWa2ycxWVHoSM1thZgNmNrBr166aBykNSlVRIhPENsZhZj8FTivx0Efc/Qf5cz4CjAC3lnmai9x9h5mdAtxtZo+6+z2lTnT3dcA6gL6+Pp/2CxABDZCLlBBb4nD3N1Z63MyuAd4CXOruJS/07r4j/32nmX0f6AdKJg7JuEolsyJSV4l0VZnZMuCDwFvd/WCZc+aaWdvYbeAy4OH6RSlBUcmsSDCSGuP4EtBGrvtps5ldD2Bmi8xsff6cU4FfmdkW4AHgR+7+42TCFRGRMYnM43D3l5c5vgNYnr/9BLCknnFJhrS3l+/aUitFZFpCqKoSqT1NGBSJjRKHiIhEosQhIiKRKHGIiEgkShwiIhKJEodkk5YKEYmNNnKSbFLJrUhs1OIQEZFIlDhERCQSJQ4REYlEiUNERCJR4hARkUiszFYYqWZmu4Ank44jggXAc0kHMQWKu37SGDMo7nqabswvcfeuak7MZOJIGzMbcPe+pOOISnHXTxpjBsVdT/WMWV1VIiISiRKHiIhEosQRhnVJBzBFirt+0hgzKO56qlvMGuMQEZFI1OIQEZFIlDgCYWafNLOHzGyzmd1lZouSjqkaZvZZM3s0H/v3zWx+0jFNxsz+wsweMbNRMwu+csbMlpnZY2b2uJl9KOl4qmFmN5vZTjN7OOlYqmVmvWb2CzPblv/7eF/SMVXDzGab2QNmtiUf9/+M/d9UV1UYzKzd3fflb/8NcI67r0o4rEmZ2WXAz919xMw+A+DuH0w4rIrM7JXAKHAD8AF3H0g4pLLMrBn4DfAmYAjYCFzl7v+eaGCTMLPXAweAb7j7uUnHUw0zWwgsdPcHzawN2AS8LQWO15KxAAADIElEQVS/awPmuvsBM5sB/Ap4n7vfH9e/qRZHIMaSRt5cIBUZ3d3vcveR/N37gZ4k46mGu29z98eSjqNK/cDj7v6Eux8FvgNckXBMk3L3e4DdSccRhbs/7e4P5m/vB7YB3clGNTnPOZC/OyP/Fev1Q4kjIGb2KTMbBP4L8LGk45mCdwJ3Jh1ExnQDgwX3h0jBxSztzGwx8EfAhmQjqY6ZNZvZZmAncLe7xxq3EkcdmdlPzezhEl9XALj7R9y9F7gVeG+y0b5osrjz53wEGCEXe+KqiTklrMSxVLRG08rM5gHfBd5f1BMQLHc/7u7nk2vx95tZrN2D2gGwjtz9jVWe+m3gR8DHYwynapPFbWbXAG8BLvVABs0i/K5DNwT0FtzvAXYkFEvm5ccIvgvc6u7fSzqeqNx9r5n9G7AMiK0wQS2OQJjZmQV33wo8mlQsUZjZMuCDwFvd/WDS8WTQRuBMM3upmc0ErgTuSDimTMoPMt8EbHP3f0o6nmqZWddYNaOZtQJvJObrh6qqAmFm3wXOIlft8ySwyt23JxvV5MzscWAW8Hz+0P2hV4OZ2Z8B1wFdwF5gs7u/OdmoyjOz5cAXgWbgZnf/VMIhTcrMbgMuJrdi67PAx939pkSDmoSZ/QnwS2Aruf+HAH/n7uuTi2pyZnYecAu5v48m4H+7+ydi/TeVOEREJAp1VYmISCRKHCIiEokSh4iIRKLEISIikShxiIhIJEocIjHIr7T6ezM7KX+/M3//JWb2YzPba2Y/TDpOkalQ4hCJgbsPAl8FPp0/9Glgnbs/CXwWeHtSsYlMlxKHSHy+AFxoZu8H/gT4PIC7/wzYn2RgItOhtapEYuLux8zsfwA/Bi7LL4suknpqcYjE63LgaSAVmxmJVEOJQyQmZnY+uZ37LgT+W36HOZHUU+IQiUF+pdWvktvT4SlyA+KfSzYqkdpQ4hCJx7XAU+5+d/7+V4CzzewNZvZL4F+BS81syMyCXZlXpBStjisiIpGoxSEiIpEocYiISCRKHCIiEokSh4iIRKLEISIikShxiIhIJEocIiISiRKHiIhE8v8BZObjQO7u9fIAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from numpy import *\n",
    "dataArr,labelMat = loadDataSet()\n",
    "weights = stocGradAscent1(array(dataArr), labelMat)\n",
    "plotBestFit(weights) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 示例：从疝气病症预测病吗的死亡率"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4-5 Logistic回归分类函数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [],
   "source": [
    "def classifyVector(inX, weights):\n",
    "    prob = sigmoid(sum(inX*weights))  # 输入Z=inX*weights（特征向量*回归系数） 计算Sigmoid值\n",
    "    if prob > 0.5:\n",
    "        return 1.0\n",
    "    else:\n",
    "        return 0.0\n",
    "\n",
    "def colicTest():\n",
    "    frTrain = open('horseColicTraining.txt')   # 打开训练集\n",
    "    frTest = open('horseColicTest.txt')   # 打开测试集\n",
    "    trainingSet = []   # 训练集列表\n",
    "    trainingLabels = []   # 训练标签列表\n",
    "    for line in frTrain.readlines():   # 读取训练文本的 每一行 \n",
    "        currLine = line.strip().split('\\t')   # 除去空格，然后用split方法 转化 成list\n",
    "        lineArr = []   # 定义行数组（处理每行数据）\n",
    "        for i in range(21):  #数据有22列，前21个为特征，最后一个是分类标签\n",
    "            lineArr.append(float(currLine[i]))  # 添加currLine的第i个元素到lineArr\n",
    "        trainingSet.append(lineArr)   # 把 lineArr 作为 子列表 添加到 训练集列表trainingSet\n",
    "        trainingLabels.append(float(currLine[21]))  #  把 currLine 最后一个元素（是标签） 添加到训练标签trainingLabels\n",
    "    trainWeights = stocGradAscent1(array(trainingSet),trainingLabels,500)   # 迭达500次 求 训练权重\n",
    "    errorCount = 0\n",
    "    numTestVec = 0.0\n",
    "    for line in frTest.readlines():   # 读取测试文本的每一行\n",
    "        numTestVec += 1.0  # numTestVec == 1.0\n",
    "        currLine = line.strip().split('\\t')  # 除去空格，然后用split方法 转化 成list\n",
    "        lineArr = []    # 定义行数组（处理每行数据）\n",
    "        for i in range(21):  # 数据有22列，前21个为特征，最后一个是分类标签\n",
    "            lineArr.append(float(currLine[i]))   # 添加currLine的第i个元素到lineArr\n",
    "        if int(classifyVector(array(lineArr), trainWeights)) != int(currLine[21]):  # 如果分类有错误\n",
    "            errorCount += 1\n",
    "    errorRate = (float(errorCount)/numTestVec)   # 计算错误率\n",
    "    print(\"the error rate of this test is: %f\" % errorRate)\n",
    "    return errorRate\n",
    "    # errorCount,numTestVec 各变量结果  25, 67.0\n",
    "\n",
    "def multiTest():    # 调用colicTest（）函数10次并求平均值\n",
    "    numTests = 10\n",
    "    errorSum = 0.0\n",
    "    for k in range(numTests):\n",
    "        errorSum += colicTest()\n",
    "    print(\"after %d iterations the average error rate is: %f\" % (numTests, errorSum/float(numTests)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "c:\\users\\dell\\appdata\\local\\juliapro-0.6.4.1\\python\\lib\\site-packages\\ipykernel_launcher.py:14: RuntimeWarning: overflow encountered in exp\n",
      "  \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "the error rate of this test is: 0.313433\n",
      "0.31343283582089554\n",
      "the error rate of this test is: 0.343284\n",
      "the error rate of this test is: 0.432836\n",
      "the error rate of this test is: 0.343284\n",
      "the error rate of this test is: 0.388060\n",
      "the error rate of this test is: 0.283582\n",
      "the error rate of this test is: 0.432836\n",
      "the error rate of this test is: 0.358209\n",
      "the error rate of this test is: 0.373134\n",
      "the error rate of this test is: 0.373134\n",
      "the error rate of this test is: 0.358209\n",
      "after 10 iterations the average error rate is: 0.368657\n"
     ]
    }
   ],
   "source": [
    "print(colicTest())\n",
    "multiTest()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
